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Abstract 


The  focus  of  this  project  was  to  assess  potential  improvements  to  the  current  wargame  replication 
system  used  by  the  Land  Force  Operational  Research  Team  (LFORT)  in  DRDC  CORA  through 
the  integration  of  human  interactors’  intentions.  The  project,  based  on  the  analysis  of  problems  in 
current  wargame  replication  systems,  reviews  competing  Artificial  Intelligence  (AI)  and  Human 
Behaviour  Representation  (HBR)  approaches,  tools  and  systems  applicable  to  the  wargame 
replication  domain.  For  each  identified  approach,  the  main  concepts,  advantages,  limitations  and 
application  areas  are  briefly  described. 

The  typical  problems  in  wargame  replications  are  categorized  widely  as  situation  /  pattern 
assessment  and  recognition,  knowledge  discovery,  decision  making  and  planning.  There  is  no 
single  AI  or  HBR  tool  that  is  appropriate  for  resolving  all  of  these  problems.  This  project 
proposes  a  framework-based  solution  by  combining  human  science  results,  existing  approaches 
and  human  behaviour  moderators  to  solve  various  problems  in  wargame  replications. 


Resume 


L’objectif  du  presente  projet  etait  d’evaluer  les  ameliorations  possibles  pouvant  etre  apportees  au 
systeme  de  replication  de  jeu  de  guerre  actuel  qu'utilise  l’Equipe  de  recherche  operationnelle  de 
la  Force  terrestre  (EROFT)  de  RDDC  CARO  a  l’aide  de  F  integration  des  intentions  des 
interacteurs  humains.  Le  projet,  en  se  basant  sur  l’analyse  de  problemes  lies  aux  systemes  de 
replication  de  jeu  de  guerre  actuels,  permet  d’examiner  des  systemes,  des  outils  et  des  approches 
possibles  d’intelligence  artificielle  (IA)  et  de  representation  du  comportement  humain  (RCH) 
applicables  dans  le  domaine  de  la  replication  de  jeu  de  guerre.  Pour  chaque  approche  identifiee, 
les  concepts,  les  avantages,  les  restrictions  et  les  secteurs  d’ application  generaux  ont  ete 
brievement  decrits. 

Les  problemes  typiques  de  la  replication  de  jeu  de  guerre  sont  places  dans  les  categories  generates 
suivantes  :  reconnaissance  et  evaluation  de  la  situation  et  de  la  tendances,  decouverte  de 
connaissances,  prise  de  decision  et  planification.  Aucun  outil  d’IA  ou  de  RCH  ne  resout 
entierement  tous  les  problemes.  Le  present  projet  propose  une  solution  fondee  sur  un  cadre  en 
combinant  les  resultats  des  sciences  humaines,  les  approches  existantes  et  les  moderateurs  de 
comportements  humains  afin  de  resoudre  les  problemes  lies  a  la  replication  de  jeu  de  guerre. 


DRDC  CORA  CR  2010-269 


This  page  intentionally  left  blank. 


11 


DRDC  CORA  CR  2010-269 


Executive  summary 


Scoping  report:  Al-Driven  wargame  replicator 

Ruibiao  Jaff  Guo;  David  Unrau;  Joe  Armstrong,  December,  2010;  DRDC  CORA 
CR  2010-269;  CAE  Professional  Services. 

Background: 

The  current  wargame  replication  system  used  by  The  Land  Force  Operational  Research  Team 
(LFORT)  in  Defence  Research  and  Development  Canada  (DRDC)  -  Centre  for  Operational 
Research  and  Analysis  (CORA)  cannot  integrate  human  interactors’  intentions  for  effective  and 
efficient  wargame  simulation  and  analysis.  To  resolve  this  shortcoming,  this  project  was  intended 
to  provide  a  review  and  definition  of  the  utility  in  developing  an  Artificial  Intelligence  (AI) 
driven  wargame  replicator. 

Objectives: 

The  overall  objectives  of  the  project  consisted  of: 

•  An  analysis  of  methodologies  to  improve  the  current  wargame  replication  system  used  by 
the  LFORT; 

•  Identification  of  potential  solutions  to  integrate  human  operator/interactor  intent  into 
wargame  replications;  and 

•  An  assessment  of  competing  AI  approaches  and  the  recommendation  of  solutions  for  AI- 
driven  wargame  replicator  systems. 

Methodology: 

Based  on  the  analysis  of  the  project  background  and  Al-driven  wargame  replication  system 
definition,  this  project  categorizes  the  AI  approaches  as  logic-based,  probabilistic,  connectionism, 
evolutionary  and  memory-based  approaches.  Logic-based  approaches  consist  of  decision  trees, 
rule-based  methods,  fuzzy  logic,  and  non-monotonic  logic.  The  typical  probabilistic  method  is 
Bayesian  Networks,  while  artificial  neural  networks  are  the  popular  method  in  connectionism, 
and  genetic  algorithms  are  the  representative  approach  of  evolutionary  computing.  Case-Based 
Reasoning  is  chosen  as  a  characteristic  method  in  memory-based  approaches.  For  the  recognized 
approach  in  related  categories  the  main  ideas,  advantages,  limitations  and  application  areas  are 
summarized  briefly. 

The  following  list  includes  a  group  of  identified  competing  AI  and  Human  Behaviour 
Representation  (HBR)  tools,  systems  and  software  for  reasoning  and  problem  solving  in  wargame 
replications: 

•  Adaptive  Control  of  Thought  -  Rational  (ACT-R); 

•  Belief-Desire-Intention  (BDI); 
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•  Connectionist  Learning  with  Adaptive  Rule  Induction  On-line  (CLARION); 

•  Man  Machine  Integrated  Design  and  Analysis  System  (MIDAS); 

•  Situation  Awareness  Model  for  Pilot-in-the-Loop  Evaluation  (SAMPLE); 

•  State,  Operator  And  Result  (Soar); 

•  Language  of  Agents  for  Modelling  Performance  (LAMP); 

•  Logic-based  software; 

•  Statistical  /  Bayesian  software;  and 

•  Neural  Networks  software. 

The  problems  in  current  wargame  replications  are  analyzed  and  categorized  as  situation/pattern 
assessment  and  recognition,  knowledge  discovery,  planning  and  decision  making.  There  is  no 
single  tool,  system  or  software  that  is  perfectly  appropriate  for  addressing  the  full-scope  of 
problems  in  wargame  replications.  Therefore,  this  project  proposes  a  generic  solution  that 
provides  the  following  features: 

•  Combines  results  from  human  science  to  enhance  reality  including  neural  networks, 
human  memory,  analogy,  dual-process  thinking  and  human  behaviour  moderators; 

•  Integrates  recognized  tools  to  save  development  time; 

•  Offers  multiple  approaches  to  enlarge  system  flexibility; 

•  Models  human  behaviour  moderators  to  characterize  variability  and  individual 
differences;  and 

•  Provides  an  approach  adaptor  to  improve  system  adaptability. 

Results: 

Finally,  the  project  recommends  two  alternative  solutions:  one  is  a  set  of  short-term  experimental 
solutions,  in  which  competing  approaches/tools  are  suggested  based  on  problem  categories  in 
wargame  replications,  and  another  is  a  mid/long-term  solution  based  on  a  unified,  integrated  and 
adaptable  architecture. 

The  proposed  mid/long-term  architecture  has  the  potential  to  be  used  for  various  Computer 
Generated  Forces  areas,  for  instance  wargame  replicators,  virtual  interactors/operators,  virtual 
adversary,  cognition  functions  such  as  situation  awareness,  decision  making,  planning  and 
knowledge  discovery,  command  and  control,  search  and  rescue,  training,  and  also  modelling  and 
simulation. 
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Sommaire 


Scoping  report:  Al-Driven  wargame  replicator 

Ruibiao  Jaff  Guo;  David  Unrau;  Joe  Armstrong,  decembre  2010;  DRDC  CORA 
CR  2010-269;  CAE  Professional  Services. 

Contexte  : 

A  ce  jour,  avec  le  systeme  de  replication  de  jeu  de  guerre  qu'utilise  l’Equipe  de  recherche 
operationnelle  de  la  Force  terrestre  (EROFT)  a  R  &  D  pour  la  defense  Canada  -  Centre  d’  analyse 
et  de  recherche  operationnelle  (RDDC  CARO),  il  est  impossible  d’integrer  les  intentions  des 
interacteurs  humains  aux  fins  de  simulation  de  jeu  de  guerre  et  d’ analyse  efficaces  et  efficientes. 
Afin  de  resoudre  ce  probleme,  le  present  projet  avait  pour  objectif  de  permettre  l’examen  et  la 
definition  de  l’utilite  de  developper  un  replicateur  de  jeu  de  guerre  axe  sur  l’intelligence 
artificielle  (LA). 

Objectif : 

Les  objectifs  generaux  du  projet  etaient  les  suivants  : 

•  Analyser  les  methodologies  afin  d’ameliorer  le  systeme  de  replication  de  jeu  de  guerre 
actuel  utilise  par  l’EROFT; 

•  Determiner  les  solutions  potentiels  permettant  d’integrer  les  intentions  des  operateurs  ou 
des  interacteurs  humains  dans  la  replication  de  jeu  de  guerre; 

•  Evaluer  les  approches  possibles  d’lA  et  recommander  des  solutions  pour  les  systemes  de 
replication  de  jeu  de  guerre  axes  sur  l’LA. 

Methodologie  : 

Selon  l’analyse  du  contexte  du  projet  et  la  definition  du  systeme  de  replication  de  jeu  de  guerre 
axe  sur  l’IA,  le  present  projet  permet  de  categoriser  les  approches  d’lA  ainsi  :  logique, 
probabiliste,  connexionniste,  evolutive  et  memorielle.  Les  approches  a  base  logique  comptent  les 
arbres  de  decision,  les  methodes  a  base  de  regies,  la  logique  floue  et  la  logique  non  monotone.  La 
methode  probabiliste  typique  est  representee  par  les  reseaux  de  Bayes,  tandis  que  les  reseaux  de 
neurones  artificiels  sont  la  methode  populaire  en  ce  qui  concerne  le  connexionnisme,  et  que  les 
algorithmes  genetiques  constituent  l’approche  representative  de  1’evolution  informatique.  Le 
raisonnement  par  cas  est  choisi  comme  methode  specifique  de  l’approche  basee  sur  la  memoire. 
Les  concepts,  les  avantages,  les  restrictions  et  les  secteurs  d’ application  generaux  sont  brievement 
resumes  pour  chaque  approche  etablie  dans  les  categories  pertinentes. 

La  liste  suivante  presente  un  groupe  d’outils,  de  systemes  et  de  logiciels  possibles  d’lA  et  de 
representation  du  comportement  humain  (RCH)  pour  le  raisonnement  et  la  resolution  de 
problemes  concernant  la  replication  de  jeu  de  guerre  : 

•  Controle  adaptatif  de  la  pensee  -  Rationalite; 
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•  Croyance,  souhait,  intention; 

•  Apprentissage  connexionniste  avec  induction  adaptative  de  regies  en  ligne; 

•  Systeme  d’ analyse  et  de  conception  integrees  homme-machine; 

•  Modele  de  connaissance  de  la  situation  pour  1’evaluation  du  pilote  dans  la  boucle; 

•  Etat  de  l’exploitant  et  resultat; 

•  Langage  des  agents  de  modelisation  des  performances; 

•  Logiciel  a  base  logique; 

•  Logiciel  bayesien  et  statistique; 

•  Logiciel  de  reseaux  de  neurones. 

Les  problemes  associes  aux  replicateurs  de  jeu  de  guerre  actuels  sont  analyses  et  places  selon  les 
categories  suivantes  :  identification  et  evaluation  de  la  tendance  et  de  la  situation,  decouverte  de 
connaissances,  planification  et  prise  de  decision.  Aucun  outil,  systeme  ou  logiciel  n’est 
entierement  capable  de  resoudre  la  portee  totale  des  problemes  lies  aux  replicateurs  de  jeu  de 
guerre.  Ainsi,  le  projet  propose  une  solution  generique  qui  offre  les  caracteristiques  suivantes  : 

•  Combine  les  resultats  des  sciences  humaines  pour  ameliorer  la  realite,  y  compris  les 
reseaux  de  neurones,  la  memoire  humaine,  1’ analogic,  la  pensee  a  deux  temps  et  les 
moderateurs  de  comportement  humain; 

•  Integre  des  outils  connus  pour  gagner  du  temps  de  developpement; 

•  Offre  de  multiple  approches  pour  elargir  1’ adaptability  du  systeme; 

•  Modelise  les  moderateurs  de  comportements  humains  pour  caracteriser  la  variability  et 
les  differences  individuelles; 

•  Fournit  l’adaptateur  d’approche  pour  ameliorer  1’ adaptability  du  systeme. 

Resultats  : 

En  terminant,  le  projet  permet  de  recommander  deux  solutions  possibles  :  la  premiere  etant  un 
ensemble  de  solutions  experimentales  a  court  terme  selon  lesquelles  des  approches  et  des  outils 
possibles  sont  suggeres  en  fonction  de  la  categoric  du  probleme  concernant  la  replication  du  jeu 
de  guerre;  et  la  deuxieme  etant  une  solution  a  moyen  et  a  long  terme  basee  sur  une  architecture 
unifiee,  integree  et  adaptable. 

L’ architecture  a  moyen  et  a  long  terme  proposee  pourrait  etre  utilisee  dans  divers  secteurs  des 
forces  generees  par  ordinateur,  par  exemple  les  replicateurs  de  jeu  de  guerre,  les  operateurs  et 
interacteurs  virtuels,  les  ennemis  virtuels,  ainsi  que  les  fonctions  cognitives  comme  la 
connaissance  de  la  situation,  la  prise  de  decision,  la  planification  et  la  decouverte  de 
connaissances,  le  commandement  et  controle,  la  recherche  et  le  sauvetage,  et  la  modelisation  et  la 
simulation. 
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1  Project  Background 


1.1  Objectives 

The  objective  of  this  report  is  to  provide  a  recommended  course  of  action  to  the  Land  Force  Operational 
Research  Team  (LFORT)  with  regard  to  increasing  the  fidelity  of  wargame  replication  for  operational 
research.  Current  wargame  replication  techniques  do  not  capture  the  intent  of  human  interactors  and 
cannot  produce  plausible  replications  of  in-field  and  ‘live’  performance  except  in  very  limited  scenarios. 
An  Artificial  Intelligence  (Al)-driven  wargame  replicator  is  envisioned  to  reduce  these  limitations.  The 
specific  objectives  of  this  report  are  as  follows: 

•  Analyze  the  current  wargame  replication  system  used  by  the  LFORT, 

•  Propose  potential  solutions  to  integrate  human  operator  intent  into  wargame  replications,  and 

•  Assess  competing  AI  approaches  and  recommend  solutions  for  Al-driven  wargame  replications. 

1.2  Scope 

The  purpose  of  this  report  is  to  provide  a  recommended  course  of  action  to  the  LFORT.  The  scope  of  this 
analysis  considered  the  three  aspects  listed  in  Section  1.1.  Namely,  based  on  the  analysis  of  the  current 
wargame  replication  system,  this  report  provides  a  review  and  makes  references  to  currently  available 
techniques  and  methods  within  the  AI  industry  for  wargame  replications.  This  report  briefly  evaluates  all 
of  the  competing  solutions  for  Al-driven  wargame  replication.  Based  on  the  assessments,  it  will  define  a 
framework  using  AI  approaches  to  represent  human  behaviour  in  wargames,  and  recommend  the  courses 
of  action.  The  domain  of  application  has  been  scoped  by  sample  use  cases,  scenarios  and  details  of  a 
prototypical  wargame  execution  provided  by  the  scientific  authority.  This  information  is  outlined  in 
Section  2,  System  definition. 

1.3  Constraints 

The  contract  start  date  is  approximately  20,  Nov,  2009.  The  deliverables  will  be  submitted  to  both  the 
scientific  authority  and  technical  authority  no  later  than  15  March,  2010.  The  scoping  report  is  a  small 
effort  and  is  co-funded  by  both  DRDC  CORA  and  CAE  PS. 

Due  to  the  limited  time  and  budget,  it  is  impossible  to  provide  a  comprehensive  review  of  AI  approaches 
and  detailed  solutions  to  the  current  problems.  Thus,  the  main  focus  of  this  project  was  the  high-level 
review  of  AI  approaches,  feasibility  and  potential  directions  of  solutions.  Detailed  approaches  and 
solutions  may  be  pursued  in  a  future  project.  Necessarily,  all  recommendations  and  scoping  are 
preliminary  in  nature,  and  subject  to  change  by  further  definition  activities. 

1.4  Assumptions 

This  section  describes  some  basic  assumptions  about  this  work.  A  modular  system  is  assumed,  based  on 
the  LFORT  way-forward  outlined  by  the  Scientific  Authority  ( Games  &  Reps  4CAE.ppt,  K.  Sprague, 
2010).  The  assumed  approach  decouples  the  interactor  (gaming),  replication  (constructive  simulation) 
and  analysis  functions  through  the  use  of  a  standard,  comprehensive  data  interchange  format  (such  as  the 
Virtual  Command  and  Control  Interface  Database,  or  ‘VDB’). 
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Also,  an  existing  constructive  or  virtual  simulation  synthetic  environment  (such  as  CAEn,  JCATS  or 
OneSAF)  is  assumed  to  be  the  basis  for  the  proposed  replicator  solution.  The  data  pertaining  to  entities 
and  tasks  related  to  scenarios  or  use  cases  in  the  synthetic  environment  should  be  accessible  to  external 
software  modules  through  standard  network  protocols,  the  High  Level  Architecture  (HLA),  the 
Distributes  Interactive  Simulation  (DIS)  protocol  or  a  fully  feature  Application  Programmer’s  Interface 
(API).  Historic  data  of  war  games  is  assumed  to  be  available  for  the  recommended  AI  approaches  to 
extract  information  and  knowledge  from  experience. 

1.5  Document  organization 

This  report  is  organized  as  follows.  Section  2  describes  the  system  definition  of  wargame  replicator 
systems.  Section  3  describes  the  technical  approach,  including  a  review  of  the  AI  approaches  based  on 
logic,  probabilistic,  connectionism,  evolutionary  and  memory-based  categories,  and  a  summary  of  the 
available  AI  and  Human  Behaviour  Representation  (HBR)  tools.  Section  4,  heavily  based  on  results  from 
the  human  sciences,  proposes  a  mid/long-term  research  and  development  system  architecture.  Section  5  is 
the  system  scoping  consisting  of  work  tasks,  milestones,  and  efforts.  The  next  section  of  the  report, 
Section  6,  presents  recommendations  for  achieving  a  functional  Al-driven  wargame  replicator,  including 
a  set  of  short-term  experiment  solutions  and  a  mid/long-term  generic  solution  for  research  and 
development.  The  last  two  sections  consist  of  acronyms  and  references. 
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2  System  definition 


2.1  High  level  concept 

A  wargame  is  a  simulated  military  operation,  carried  out  to  test  the  validity  of  a  war  plan  or  operational 
concept1  (Sprague  &  Dobias,  2010).  Wargame  replication  refers  to  the  process  of  repeating  the  execution 
of  a  specific  wargame  many  times  to  enable  statistically  valid  analysis  in  an  environment  that  can  be 
sensitive  to  the  impact  of  random  factors,  such  as  the  influence  of  a  ‘lucky  shot’  (Sprague,  2010). 
LFORT  uses  computer  based  simulation  as  the  basis  for  wargaming.  With  simulation  tools,  such  as  the 
Close  Action  Environment  (CAEn),  human  interactors  can  control  military  forces  and  enact  simulated 
military  operations.  However,  military  subject  matter  experts  are  a  limited  resource,  so  the  number  of 
replications  that  can  be  conducted  with  human  interactors  is  very  limited. 

Computer  driven  control  of  the  military  forces  (often  referred  to  as  ‘constructive  simulation’)  would 
enable  an  unlimited  number  of  replications  to  be  conducted.  However,  sophisticated  Artificial 
Intelligence  (AI)  techniques  are  required  in  a  fully  autonomous  constructive  simulation  process  to  ensure 
that  the  intent  of  the  human  interactors  is  captured,  and  replicated  with  sufficient  fidelity  during  the 
computer  driven  replications. 

The  high  level  concept  for  LFORT  wargame  replication  is  as  follows: 

•  Human  interactors  control  synthetic  equipment  in  an  interactive  simulation  environment 
(‘computer  game’)  to  execute  a  number  of  runs  of  the  experimental  scenario; 

•  The  human  interactions  with  the  gaming  environment  and  the  activities  in  the  gaming 
environment  are  logged  to  an  authoritative,  standardized  data  repository; 

•  If  required,  further  information  is  elucidated  from  the  human  interactors,  potentially  in  an 
interview  process,  and  this  information  is  added  to  the  record  of  the  wargame  execution; 

•  If  required,  analysis  is  performed  on  the  information  recorded  from  the  wargame  execution  and 
the  results  of  this  analysis  are  also  added  to  the  record  of  the  wargame  execution; 

•  An  Al-based  replicator  makes  use  of  the  record  of  the  human-interactor  driven  wargames  to 
capture  the  intent  of  the  human  interactors  and  produce  meaningful,  constructive  replications  of 
the  scenario  to  the  required  level  of  statistical  validity; 

•  The  Al-based  replicator  records  the  events  and  outcomes  of  the  replications  in  the  same 
repository  in  the  same  format  as  the  human  interactor  based  executions;  and 

•  Analysis  is  performed  to  validate  the  plausibility  of  the  computer  driven  replications  as  compared 
to  the  human  driven  executions,  and  to  extract  the  experimental  outcomes  from  both  the  human 
and  computer  driven  replications. 

2.2  Wargame  replicator  requirements 

There  are  two  main  dimensions  to  the  wargame  replicator  requirements.  The  first  dimension  is  the  nature 
of  the  wargames  that  need  to  be  replicated.  The  second  dimension  is  the  information  that  needs  to  be 
derived  from  the  replications  themselves.  Example  experiments  outline  the  nature  of  the  wargames  that 
need  to  be  replicated.  Interviews  with  interactors  around  some  example  scenarios  provide  indications  on 


1  Based  on  Random  House  Dictionary. 
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the  aspects  of  human  behaviour  that  might  need  to  be  modelled  in  the  wargame  replications.  Section 
2.2.1  outlines  an  example  experiment  and  Section  2.2.2  outlines  an  example  scenario  only  (no 
analysis/results).  Detailed  actions  and  decision  making  information  were  not  recorded  for  the  example 
experiment  (Section  2.2.1),  thus  only  a  summary  of  results  appears.  It  is  a  useful  example  in  that  it 
describes  a  problem  to  be  addressed  via  wargaming,  the  basic  simulation  environment  and  entities,  and 
the  recommended  ‘solution’  based  on  the  results.  On  the  other  hand,  the  example  scenario  (Section  2.2.2) 
includes  a  record  of  decision  making  for  the  forces  involved,  but  no  analysis  or  interpretation  of  the 
results.  It  is  useful  in  that  it  provides  a  glimpse  into  the  reasons  behind  the  actions  taken  in  the  game  and 
also  the  various  factors  that  contribute  to  the  success  or  failure  or  the  ‘virtual’  mission.  The  information 
was  provided  by  DRDC  CORA  and  is  included  for  context  and  reference.  Section  2.2.3  outlines  some  of 
the  replication  requirements  of  the  wargame  replicator. 


2.2.1  Example  experiment 

The  example  experimental  matrix  consists  of: 

•  Two  vehicles  modeled:  ‘ALPHA’  and  ‘BETA’; 

•  Two  weapon  stations:  1-man  turret  with  remote  capability,  and  a  remote  weapon  station  (RWS); 
and 

•  Five  weapon  systems:  .50  cal  Heavy  Machine  Gun  (HMG)  ;  40  mm  Automatic  Grenade 
Launcher  (AGL);  Antitank  Guided  Missile  (ATGM);  20  mm  cannon;  and  a  Low  Velocity  (LV) 
25  mm  cannon  (notional). 

There  are  three  scenarios  in  the  experiment: 

•  Irregular  Warfare  (IW); 

o  Scenario  1:  Convoy  ambush  in  urban  terrain;  and 

o  Scenario  2:  Recce  picket  objective  of  a  town  in  mixed  urban/open  terrain. 

•  Conventional  Warfare  (CW); 

o  Scenario  3:  Fighting  Withdraw  in  open  terrain 

The  experimental  matrix  of  wargames  played  is  shown  in  Table  1.  The  ‘No.  Variations’  column  is  derived 
from  the  listed  entries  in  the  previous  three  columns  as  the  following  product:  (number  of  weapon 
stations)  x  (number  of  vehicles)  x  (number  of  scenarios).  The  ‘No.  Replays’  column  records  the  number 
of  games  that  the  interactors  played  per  variation  (each  variation  represents  one  possible  equipment 
option  to  be  tested).  The  ‘No.  Games’  column  records  the  total  number  of  interactor  games  played  for  the 
entire  set  of  variations,  and  the  ‘No.  Reps’  column  records  the  total  number  of  replications  generated  for 
the  set  of  variations. 
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Table  1:  Wargames  Played 


ID 

Priority 

Weapon  System 

Weapon 

Stations 

Vehicles 

Scenarios 

No. 

No. 

No. 

No. 

Variations 

Replays 

Games 

Reps 

1 

HIGH 

LV  25mm  /  C6 

RWS,  1-man 

Vehicle  A,  Vehicle  B 

I,  II  and  III 

12 

4 

48 

960 

2 

MED 

LV  25mm  /  C6  / 
2xATGM 

RWS,  1-man 

Vehicle  A,  Vehicle  B 

II  and  III 

8 

4 

32 

640 

3 

HIGH 

AGL / C6 

RWS,  1-man 

Vehicle  A,  Vehicle  B 

I,  II  and  III 

12 

4 

48 

960 

4 

MED 

AGL  /  C6  /  2xATGM 

RWS,  1-man 

Vehicle  A,  Vehicle  B 

II  and  III 

8 

4 

32 

640 

5 

HIGH 

.50  cal  /  C6 

RWS,  1-man 

Vehicle  A,  Vehicle  B 

I,  II  and  III 

12 

4 

48 

960 

6 

MED 

.50  cal  /  C6  / 
2xATGM 

RWS,  1-man 

Vehicle  A,  Vehicle  B 

II  and  III 

8 

4 

32 

640 

7 

HIGH 

C6  /  4xATGM 

RWS,  1-man 

Vehicle  A,  Vehicle  B 

II  and  III 

8 

4 

32 

640 

8 

HIGH 

25mm  /  C6 

2-man 

Vehicle  BASELINE 

I,  II  and  III 

3 

4 

12 

240 

9X 

HIGH 

20mm  /  C6 

RWS,  1-man 

Vehicle  A,  Vehicle  B 

I,  II  and  III 

12 

4 

48 

960 

HIGH  PR.  TOTALS  - 

59 

4 

236 

4720 

MED  PR.  TOTALS  - 

24 

4 

96 

1920 

To  the  accuracy  gamed,  the  overall  results  for  all  three  scenarios  indicate  that: 

•  The  weapon  system  was  the  BIGGEST  factor; 

•  The  vehicle  only  made  a  noticeable  difference  in  Scenario  1 ;  and 

•  The  weapon  station  did  not  matter. 

The  specific  results  for  the  individual  scenarios  are  now  provided. 

Figure  1  is  the  visual  view  of  Scenario  1.  The  Measures  of  Effectiveness  (MOEs)  for  that  scenario  are  as 
follows: 

•  Convoy  Survival  Rate:  50%; 

•  BLUE  Residual  Combat  Strength:  25%;  and 

•  RED  Losses:  25%. 

The  results  of  Scenario  1  are  summarized  below: 

•  BETA  faired  better  then  ALPHA; 

•  In  most  cases,  the  weapon  stations  did  not  make  much  of  a  difference; 

•  The  LV25mm,  HMG  and  AGL  each  logged  (high-ranking)  performances  comparable  to  the 
25mm  cannon  of  the  BASELINE  vehicle;  and 

•  The  arming  distance  for  the  AGL  was  a  negative  factor  for  that  weapon  in  close  combat. 


Figure  1:  Example  Wargame:  Scenario  1 
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Figure  2  is  the  visual  view  of  Scenario  2,  with  the  following  MOEs: 

•  Identification  of  RED  Units:  50%; 

•  BLUE  Residual  Combat  Strength:  25%;  and 

•  RED  Losses:  25%. 

The  results  of  Scenario  2  are  summarized  below: 

•  ALPHA  and  BETA  performances  were  comparable  to  one  another; 

•  In  most  cases,  the  weapon  stations  did  not  make  much  of  a  difference; 

•  On  average,  1-man  turret  engagements  occurred  at  longer  ranges  than  RWS  engagements;  and 

•  ATGM  options  logged  performances  comparable  to  the  BASELINE,  with  the  LV25mm  not  far 
behind. 


Figure  2:  Example  Wargame:  Scenario  2. 


Figure  3  is  the  screen  shot  of  Scenario  3  with  the  following  MOEs: 

•  70%  BLUE  Residual  Combat  Strength;  and 

•  30%  of  RED  Losses. 


Rather  than  list  the  results  of  this  final  scenario,  the  results  of  Scenario  3  are  shown  in  Figure  4  as  they 
might  be  presented  to  the  military  sponsor  of  the  wargame. 
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Figure  3:  Example  Wargame:  Scenario  3 


Overall 

ALPHA  and  BETA  comparable  to  one  another 

In  most  cases,  weapon  stations  did  not  make  much  of  a 
difference 

ATGM  logged  performances  far  superior  to  the  baseline. 
All  other  weapons  more  or  less  equally  bad 
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Figure  4:  Example  Wargame:  Scenario  3  results 
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Finally,  this  example  has  the  following  overall  recommendations,  based  on  all  three  scenarios: 

•  Best  combination  overall:  NONE  were  high  performers  in  all  three  of  the  scenarios 

•  Need  to  combine  ATGMs  with  LV25mm  to  maximize  performance  over  the  three  scenarios: 
BETA  vehicle  preferred  on  the  basis  of  Scenario  1  (either  station);  Require  more  than  a  wargame 
to  inform  vehicle  selection  though  (CAEn  vehicle  templates  are  not  complex  enough). 

2.2.2  Example  scenario 

The  following  example  scenario  (Sprague,  2010)  was  used  to  elucidate  information  on  the  decision 
making  process  from  the  interactors.  This  section  outlines  the  scenario  and  the  feedback  from  the 
interactors.  ScenarioCutoff_RUN_l.doc  (Sprague.  2010)  details  the  scenario. 

The  scenario  is  based  in  an  urban/open  terrain  with  a  playing  area  of  10km  by  10km.  Blue  force  has  been 
tasked  with  clearing  the  village.  Blue  force  commander’s  intent  is  to  destroy  or  capture  all  members  of 
Red  force  to  remove  the  current  and  future  threat.  Red  force’s  intent  is  to  maximize  Blue  force  casualties, 
create  maximum  confusion  and  to  escape  to  fight  in  the  future.  The  terrain  is  a  mainly  flat  urban  area 
with  some  rolling  hills.  The  village  mainly  consists  of  two  story  buildings  with  some  larger  buildings, 
trees  and  shrubs.  There  is  a  paved  road  network. 

Blue  force  is  a  light  infantry  section  of  eight  (8)  soldiers  with  C7  rifles.  Red  force  is  composed  of  eight 
(8)  soldiers  who  have  trained  together  as  a  formed  group  and  have  a  well-defined  set  of  tactics,  techniques 
and  procedures  (TTPs)  that  they  follow.  Red  force  is  armed  with  C7  rifles. 

Subject  matter  expert  interactors  executed  this  scenario  and  recorded  the  details  of  their  decision-making 
processes  during  the  scenario  execution.  Examples  of  this  information  are  listed  in  the  following  tables 
(Table  2)  and  are  provided  for  reference.  The  text  is  drawn  from  the  Blue  force  commander  in  the 
scenario,  and  records  a  series  of  seven  (7)  major  decision  points  over  a  span  of  approximately  nine  (9) 
minutes  of  game  time.  After  a  series  of  weapons  fire  exchanges  by  a  stationary  Blue  force  on  a  moving 
Red  force,  by  the  time  of  the  last  entry  (ID#7),  all  Red  units  have  been  destroyed.  However,  it  is  not  clear 
to  Blue  whether  or  not  there  are  still  Red  units  at  large  since  they  rely  on  visual  detections  of  an  enemy 
that  is  constantly  changing  its  position  amidst  rubble,  trees  and  nearby  buildings. 

Note  that  this  particular  scenario  is  at  least  reasonably  robust  under  slight  variations  in  decision-making, 
since  it  basically  describes  a  ‘shoot-out’  along  well-defined  positions  and  paths  of  motion.  Case  in  point, 
wargames  currently  have  to  pay  close  attention  to  how  a  game  will  be  replicated  in  order  to  achieve 
credible  results.  That  is,  the  replicator  capability  constrains  the  complexity  of  scenarios  that  can  be  gamed 
and  analyzed  statistically. 
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Table  2:  Decision  making  feedback  from  SME  Interactors 


ID 

TIME 

SITUATION 

GOAL 

CONSIDERATIONS  / 
COAs 

DECISION 

ACTION 

RESULT 

1 

00:00 

BLUE  Section 
Deployed  as  cutoff 
team  for  RED  forces 
fleeing  from  building. 

Kill  /  stop  RED  from 
fleeing  building 

Range,  approx. 

300m; 

RED  has  good  cover 
to  withdraw; 

BLUE  also  has  good 
cover  at  location  but 
little  freedom  of 
movement; 

BLUE  has  little  cover 
in  the  immediate 

area; 

Engage  RED  as 
they  become 
visible. 

Engage  RED 
with  Direct 

Fire; 

Use  cover  to 
alternate 
between  being 
under  cover  and 
firing  on  RED 
via  Assault 
Groups. 

4  RED  seen 
exiting  building 
to  covered 
positions  in 
front  of 
building; 

BLUE  engaged 
but  no  RED  are 
believed  to  have 
been  killed. 

2 

00:39 

Half  Section  of  RED 
seen  exiting  Building 
and  have  taken  up 
positions  in  front  of 
building  (as  per 
results  in  #1). 

As  above  (#1). 

RED  now  outside 
building  under  less 
cover; 

RED  now  engaging 
BLUE. 

As  above. 

As  above. 

A  second  half¬ 
section  exits  the 
building; 

0  believed 
killed  in  action 
(KIA) 

3 

01:43 

2  BLUE  KIA  in  same 
assault  group; 

2  half-sections  of 

RED  now  outside 
building  using  cover; 
RED  continues  to 
engage  BLUE. 

As  above  (#2). 

Firepower  in  section 
is  unevenly  divided 
between  assault 
groups. 

As  above. 

Use  individual 
cover/fire 

movement 

instead  of 
assault  groups. 

RED  is  using 
fire  and 

movement  to 

withdraw 
towards  woods; 

0  RED  believed 
KIA. 

ID 

TIME 

SITUATION 

GOAL 

CONSIDERATIONS  / 
COAs 

DECISION 

ACTION 

RESULT 

4 

05:20 

RED  using  fire  and 
movement  to 

withdraw  towards 
woods  (as  per  results 
in  #3) 

As  above  (#3); 

Plus  halt  RED 
withdrawal. 

RED  difficult  to 
detect; 

BLUE  Direct  Fire  has 
been  ineffective. 

Use  Suppressive 

Fire  to  halt  RED 
withdrawal. 

Put  half- section 
on  Suppressive 
Fire; 

Target  likely 

RED  position  at 
initial  cover  just 
outside 
building. 

RED 

withdraws; 

Continues; 

1  RED  believed 

KIA 

5 

07:02 

RED  continues 
withdrawing  towards 
woods. 

As  above  (#4). 

As  above  (#4). 

As  above  (#4). 

Shift 

Suppressive 

Fire  to  follow 
the  RED 
withdrawal. 

RED 

withdrawal 

continues. 

6 

07:45 

As  above  (#5). 

As  above  (#5). 

RED  moving  across 
more  open  terrain; 
Visibility  improves. 

Use  Direct  Fire  to 
halt  RED 
withdrawal. 

Place  all  BLUE 
on  Direct  Fire; 
Engage  RED 
when  seen. 

3  additional 

RED  believed 
KIA; 

RED  continue 
to  withdraw; 

7 

09:18 

Remaining  RED 
forces  continue  to 
withdraw  using  fire 
and  movement. 

As  above  (#6). 

Right-most  BLUE 
soldier  has  best 
visibility  of  RED; 

RED  easily  seen  and 
moving  across  open 
terrain  along  final 
stretch  to  woods. 

Shift  BLUE  section 
to  better  vantage 
point  to  engage 

RED. 

Use  fire  and 

movement  to 

shift  half  of 
BLUE  section 
to  the  right; 
Continue  Direct 
Fire. 

Majority  of 

RED  KIA; 
Shifting  section 
proved 

unnecessary  as 
RED  becomes 
visible  to  all  of 
BLUE  as  soon 
as  they  cross  the 
road. 

2.2.3  Replication  requirements 


The  goal  of  the  replicator  is  to  allow  the  execution  of  an  experimental  run  to  be  repeated  without  human 
interactors  in  a  fashion  that  is  consistent  with  the  execution  of  the  scenario  with  human  interactors.  This 
is  necessary,  as  military  operations  deal  with  many  unpredictable  factors  that  are  necessarily  modelled  by 
random  processes.  For  instance,  weapon  accuracy  and  weapon  effects  are  normally  modelled  using 
probability  tables.  The  outcomes  of  a  specific  scenario  are  critically  sensitive  to  these  probabilistic 
factors.  For  instance,  the  survival  of  a  vehicle  crossing  open  ground  will  be  highly  dependant  on  whether 
it  is  detected  by  enemy  forces,  whether  it  is  engaged  by  enemy  weapons  and  how  it  is  damaged  by 
weapon  impacts.  The  future  actions  of  the  force  are  in  this  fashion  strongly  influenced  by  the  critical 
outcomes  of  a  series  of  events,  many  of  which  are  random  in  nature. 

Thus,  to  measure  the  effectiveness  of  a  piece  of  equipment,  or  a  specific  tactic,  many  runs  across  a 
number  of  scenarios  will  have  to  be  executed  to  produce  measurable  significance  across  the  many  random 
factors.  This  is  the  requirement  for  repetition.  The  replicator  is  complicated  by  the  fact  that  the  execution 
of  the  scenario  is  also  sensitive  to  the  intent  of  the  human  operators,  which  may  vary  in  time  as 
circumstances  change.  For  instance,  a  force  commander  may  order  an  action  that  has  a  low  probability  of 
destroying  a  resource.  However,  if  the  resource  is  then  destroyed;  the  force  commander  may  completely 
re -plan  future  actions  and  employ  different  tactics  or  strategies  than  if  the  resource  was  not  destroyed. 

If  warfighter  interactors  are  used  to  provide  this  control  (in  an  interactive  simulation  or  ‘gaming’  mode  of 
experimentation),  the  number  of  executions  that  can  be  performed  are  strictly  limited  based  on  resource 
availability.  This  is  the  Artificial  Intelligence  (AI)  requirement  of  the  replicator.  The  wargame  replicator 
is  required  to  simulate  the  intent  of  the  human  interactors  to  produce  executions  that  are  consistent  with 
the  operations  of  human  inter  actors  in  the  gaming  environment. 

2.3  Wargame  replicator  concept 

Current  wargame  replication  systems  do  not  take  into  account  the  intent  of  the  inter  actors  and  thus 
courses  of  action  taken  by  system-controlled  entities  in  a  replication  may  vary  greatly  from  actions  that 
would  have  been  taken  by  interactor-controlled  entities  under  the  same  set  of  circumstances.  Therefore, 
wargame  replication  needs  to  somehow  incorporate  the  intent  of  the  wargame  players  in  a  meaningful  and 
efficient  way,  providing  a  more  plausible  representation  of  the  behaviours  of  both  the  virtualized  players 
and  the  agents  within  these  wargame  simulations. 

Figure  5  is  the  proposed  future  architecture  of  the  Post-Gaming  Analysis  Tool  (PGAT+)  from  DRDC 
CORA.  CAEn  and  JCATS  (‘Joint  Conflict  and  Tactical  Simulation’)  are  synthetic  environments.  The 
VDB  (Virtual  Data  Base)  will  store  all  results  of  wargame  executions  for  use  in  three  tools:  the  Post- 
Wargame  Analysis  Tool,  Aramis  (a  wargame  viewing  environment2)  and  a  Replication  Query  Tool. 


2  Aramis  is  a  product  of  Simfront  Corporation.  See  http://www.simfront.com. 
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Figure  5:  Post-Gaming  Analysis  Tool  (PGAT) 

Critically,  the  VDB  is  used  as  the  central  data  repository  to  decouple  interactor  driven  executions  from 
replicated  executions  from  post  game  analysis.  All  systems  will  read  from  the  VDB  and  write  from  the 
VDB  to  gain  the  inputs  they  require  and  to  store  the  outputs  they  produce.  In  this  fashion,  the  input 
requirements  for  the  wargame  replicator  become  data  requirements  for  what  information  must  exist  in  a 
VDB  in  order  to  produce  replication  for  a  specific  experimental  plan.  These  data  requirements  may  be 
satisfied  by  a  combination  of: 

•  Interactor  drive  scenario  executions  that  produce  data  stored  in  the  VDB; 

•  Interactor  interviews  that  supplement  data  stored  in  the  VDB ;  and 

•  Analysis  and  queries  that  produce  analysis  results  from  data,  and  store  these  results  as  further  data 
in  the  VDB. 

The  Al-driven  wargame  replicator  can  integrate  human  interactors’  intent  into  the  replications  (as  shown 
in  Figure  6). 


The  AI  Replication  Engine  integrates  the  intent  of  human  interactors  into  wargame  replications.  It 
acquires  data  of  the  initial  wargame  from  the  PGAT  database,  combines  the  data  with  human  intentions  to 
form  new  inputs  of  the  wargame  replications.  The  human  intent  may  be  tasks  related  to  situation 
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assessment,  planning,  decision  making  and  knowledge  discovery.  Note  that  additional  data,  not 
necessarily  stored  in  the  VDB,  may  be  required  to  run  replications.  For  example,  standardized  probability 
tables  for  detections  and  weapon  accuracy/lethality  are  also  required  as  inputs.  These  data  do  not 
currently  reside  in  the  VDB.  Nor  do  the  interview  data.  Also,  quite  possibly,  the  results  of  certain  queries 
may  lie  outside  the  current  array  of  fields  supported  by  the  VDB.  In  any  case,  the  VDB  will  have  to  be 
extended  or  supplemented  in  order  to  support  replication.  For  the  remainder  of  this  document,  we  assume 
the  former  (extended). 
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3  Technical  approach 


This  section  reviews  the  available  approaches  and  tools  of  Artificial  Intelligence  (AI)  and  Human 
Behaviour  Representation  (HBR)  and  proposes  a  unified  human  science  oriented  AI  solution  for  problem 
solving  in  wargame  replications. 

3.1  AI  approaches 

AI  approaches  may  be  roughly  categorized  as  logic-based,  probabilistic,  connectionism,  evolutionary  and 
memory-based  approaches. 

3.1.1  Logic-based  approaches 

Logic  is  the  study  of  reasoning  that  is  used  in  most  intellectual  activity.  Symbolic  logic  is  one  area  of 
logic  that  studies  the  purely  formal  properties  of  a  string  of  symbols.  There  are  also  many  branches  of 
symbolic  logic,  such  as  propositional  logic,  predicate  logic,  temporal  logic,  modal  logic,  fuzzy  logic  and 
non-monotonic  logic.  However,  propositional  logic  and  predicate  logic  are  two  main  subfields.  One  of 
logic’s  powerful  features  is  its  reasoning  ability.  It  is  through  its  deduction  power  that  logic  is  the 
foundation  of  many  artificial  intelligence  approaches.  This  section  summarizes  several  typical  logic-based 
approaches  that  are  very  useful  for  problem  solving  in  Al-driven  wargame  replications,  including  decision 
trees,  rule -based  systems,  fuzzy  logic  and  non-monotonic  logic. 

A  decision  tree  (Howard,  1966)  is  a  directed  logic  structure  with  the  topology  of  a  free.  It  is  a  decision 
support  tool  that  uses  a  tree-like  graph  or  model  of  decisions  and  their  possible  consequences.  Decision 
trees  have  been  used  in  many  applications  areas,  for  instance,  decision  making  (Gensure,  2007;  Santos,  et 
al.,  2008),  knowledge  discovery  (Zhang  et  al.,  1999;  Hsu  &  Wang.,  2005),  and  prediction  (East  & 
Sharfstein,  2006).  In  the  Al-driven  wargame  replicator,  decision  trees  may  be  used  in  problems  associated 
with  decision  making,  pattern  discovery  and  planning. 

A  rule-based  system  is  a  knowledge -based  system  that  uses  IF-THEN  rules  to  represent  knowledge 
(Brownston  et  al.,  1985;  Cooper  &  Wogin,  1988).  Rule-based  systems  are  a  relatively  simple  approach 
that  can  be  adapted  to  a  great  number  of  problems  (Russell  and  Norvig,  2003;  Anderson  et  al.,  2004, 
Ligeza,  2006).  The  rule -based  system  itself  uses  a  simple  technique:  it  starts  with  a  rule -base,  which 
contains  all  of  the  appropriate  knowledge  encoded  into  IF-THEN  rules,  and  a  working  memory,  which 
may  or  may  not  initially  contain  any  data,  assertions  or  initially  known  information.  Rules  are  of  the 
form:  IF  some  condition  THEN  some  action.  There  is  an  inference  engine  that  derives  action  based  on  the 
conclusions  obtained  from  the  rule -base. 

Rule -based  systems  have  been  used  in  great  number  of  application  areas,  for  example, 

•  Situation  assessment  and  threat  evaluation  (Pew  and  Mavor,  1998;  De  Jongh,  et  al.,  1994; 
Lightfoot,  2003;  Duke  et  al.,  1989;  Tang  &  Zhang,  2009); 

•  Strategic  planning  (Huang,  2009;  Liao  et  al.,  2003); 

•  Decision  support  (Eom  et  al.,  1998,  Eom  and  Kim,  2005;  Metaxiotis,  et  al.,  2004;  Momoh  et  al., 
1994); 

•  Simulating  the  game  strategies  in  computer  games  (  Rielly,  2005); 
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•  Expert  systems  (Lindsay  et  al.,  1980;  Buchannan  &  Shortliffe,  1984);  and 

•  Natural  language  processing. 

In  the  Al-driven  wargame  replicator,  rule-based  systems  can  be  used  in  various  levels  from  situation 
assessment,  threat  detection  to  decision  and  planning. 

Fuzzy  reasoning  is  based  on  fuzzy  set  theory  and  fuzzy  logic  (Zadeh,  1965,  1968;  Kosko,  1993)  to 
specify  how  well  an  object  satisfies  a  vague  description.  The  main  advantage  of  fuzzy  reasoning  is  the 
ability  to  mimic  human  decision  making  to  handle  vague  or  imprecise  or  imperfect  concepts. 

Fuzzy  reasoning  also  has  wide  usages,  for  instance,  situation  awareness  in  military  applications  (Chai,  et 
al.,  2007;  Jeppesen  and  Trellue,  1997;  Rao,  et  al.,  2008;  Liang,  2007;  Gonsalves  et  al.,  2000), 
reconnaissance  (Ragsdale  et  al.,  1997),  planning  (Kewley  &  Embrechts,  2002),  optimal  strategy  selection 
to  respond  to  a  threat  (Smith  111,  2002),  fuzzy  control  for  UAV,  robots,  aircraft  flight  (Guo  et  al.,  2008), 
electronic  equipment,  etc.  In  Al-driven  wargame  replications,  fuzzy  reasoning  may  be  used  for  situation 
and  target  value  evaluation,  reconnaissance,  and  strategy  or  tactics  selection. 

Non-monotonic  logic  is  a  formal  logic  whose  consequence  relation  is  not  monotonic.  Most  formal  logics 
have  a  monotonic  consequence  relation,  meaning  that  adding  a  formula  to  a  theory  never  produces  a 
reduction  of  its  set  of  consequences.  Some  recognized  non-monotonic  reasoning  models  include  the 
Default  Reasoning  (Reiter,  1980),  the  Abductive  Reasoning  (Peirce,  1958),  the  Reasoning  about 
Knowledge  (Moore,  1984,  1985)  and  the  Belief  Revision  (Gardenfors  and  Rott,  1995).  Some  applications 
of  non-monotonic  reasoning  consist  of  situation  assessment  (Ly  et  al.,  2003),  air  force  threat  correlation 
(Cohen  and  Laskey,  1986),  qualitative  physics,  databases,  learning,  logic  programming,  diagnosis  or 
robotics  (Etherington  and  Kautz,  1994;  Nunez  et  al.,  2007),  and  cognitive  functions  (Novak,  2008).  In  an 
Al-driven  wargame  replicator,  non-monotonic  logic  is  a  relevant  advanced  topic.  It  can  be  combined  with 
rule -based  systems  to  support  deeper  problem  solving  in  wargame  replications. 

The  advantages  of  logic-based  system  are  as  follows: 

•  Sound  logic  foundation; 

•  Able  to  represent  expert’s  knowledge  and  thinking  patterns; 

•  Natural  knowledge  representation,  i.e.,  an  IF-THEN  rule  is  like  “In  such-and-such  situation,  I  do 
so-and-so”  in  human  thinking;  and 

•  Separation  of  knowledge  from  its  processing. 

Logic-based  systems  also  have  some  limitations,  for  example,  requiring  explicit  expert’s  knowledge  and 
thinking  patterns,  difficult  knowledge  acquisition,  lack  of  common  sense  needed  in  some  decision 
making,  opaque  relations  between  rules,  and  ineffective  exhaustive  search  strategy. 

3.1.2  Probabilistic  approaches 

The  basic  idea  of  probabilistic  reasoning  is  that  real  world  phenomena  can  be  reasonably  modeled  as 
probability.  Handling  uncertainty  is  the  focus  of  probabilistic  reasoning.  The  foundation  of  probabilistic 
reasoning  is  the  well-known  Bayes  Theorem  (Bayes,  1763)  that  shows  how  one  conditional  probability 
depends  on  given  evidence.  The  key  idea  is  that  the  probability  of  hypothesis  H  given  event  e  (evidence) 
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depends  not  only  the  relationship  between  H  and  e  blit  on  the  absolute  probability  of  H  independent  of  e, 
i.e.: 


Pin  k)  = 


Pie\H)P(H) 


where  P(H)  is  the  prior  probability  or  marginal  probability  of  H,  P(H  I  e)  is  the  conditional  probability  of 
H,  given  e,  P(e  I  H)  is  the  condition  probability  of  e  given  PI,  and  P(e)  is  the  prior  probability  or  marginal 
probability  of  e. 

A  Bayesian  network,  derived  from  Bayes  theorem,  is  a  probabilistic  graphical  model  that  represents  a  set 
of  random  variables  and  their  conditional  independencies  via  a  Directed  Acyclic  Graph  (DAG)  (Pearl, 
1988;  Murphy,  1998;  Russell  &  Norvig,  2003).  It  provides  a  natural  tool  for  dealing  with  two  problems 
that  occur  throughout  applied  mathematics  and  engineering  -  uncertainty  and  complexity  -  and  in 
particular  they  are  playing  an  increasingly  important  role  in  the  design  and  analysis  of  machine  learning 
algorithms. 

The  main  advantages  of  Bayesian  networks  include: 

•  Supporting  decision  making; 

•  Readily  handling  uncertain  information  or  incomplete  data  sets; 

•  Able  to  handle  different  variable  types  or  different  sources  of  knowledge,  e.g.  subjective  beliefs 
and  empirical  data; 

•  Readily  facilitating  use  of  prior  knowledge  or  combining  expert  knowledge; 

•  A  clear  semantic  interpretation  of  the  model  parameters; 

•  Capable  of  learning  about  causal  relationships;  and 

•  Flexible  applicability  for  different  levels  of  same  problem  domain. 


Bayesian  networks  also  have  some  limitations,  for  instance: 

•  Quality  depending  on  the  quality  of  the  prior  beliefs  or  model; 

•  No  feedback  loop; 

•  Difficult  to  examine  the  solutions  of  the  networks; 

•  NP-hard  calculation  costs;  and 

•  Difficult  to  get  the  probability  knowledge  in  some  domains. 


Bayesian  belief  networks  have  numerous  applications  in  various  areas,  including  situation  awareness, 
(Johansson  and  Falkman,  2008;  Mahoney  et  al.,  2002;  Wright  et  al.,  2002;  Baumgartner  et  al.,  2008), 
threat  assessment/detection  (Ghazi  et  al.,  2004;  Suzic,  2005),  decision  making  (Russell  &  Norvig,  2003; 
Starr  &  Shi,  2002;  Watthayu  &  Peng,  2004;  Brynielsson,  2006),  planning  (Vaccaro  &  Guest,  2004), 
prediction  of  terrorist  attack  (Jha,  2009),  pattern  recognition,  Modelling  and  Simulation  (M&S), 
knowledge  discovery,  data/information/text  fusion  and  mining  (NASA),  gaming,  speech  recognition, 
engineering  (NASA,  General  Electric,  Lockheed  Martin,  Hewlett-Packard  (HP),  Intel,  American 
Airlines),  information  technology  (Microsoft,  NASA,  etc.),  computational  biology  &  bioinformatics, 


DRDC  CORA  CR  2010-269 


17 


medical  diagnosis  (BiopSys,  Microsoft)  &  medical  image  process,  and  law.  The  AI  wargame  replicator 
can  use  Bayesian  inference  at  various  levels,  e.g.  situation  awareness,  threat  detection  and  evaluation, 
strategy  and  tactics  selection,  decision  making  and  planning. 

Other  approaches  associated  with  probability  computing  include  Naive  Bayes  (Bhargavi  &  Jyothi,  2009; 
Marhav,  2002;  Chia  &  Williams,  2003;  Galli  et  al.,  2009),  and  hidden  Markov  model  (HMM)  (Rabiner, 
2989;  Marhav,  2002;  Cappe  et  al.,  2005;  Popken  &  Cox,  2003;  Andersson  ,  2003,  Kelley  et  al.,  2008; 
Inamura  et  al.,  2006).  In  addition,  a  kernel  computing-based  approach,  known  as  support  vector  machines 
(SVM)  (Vapnik,  1995;  Burges,  1998;  Cristianini  &  Shawe -Taylor,  2000)  is  also  a  useful  method  for 
classification  and  regression. 

3.1.3  Connectionism  approaches 

Connectionism  is  a  set  of  approaches  in  the  fields  of  artificial  intelligence,  cognitive  psychology, 
cognitive  science,  neuroscience  and  philosophy  of  mind,  which  models  mental  or  behavioral  phenomena 
as  the  emergent  processes  of  interconnected  networks  of  simple  units.  There  are  many  forms  of 
connectionism,  but  the  most  common  forms  use  artificial  neural  network  models. 

An  Artificial  Neural  Network  (ANN)  (McCulloch  and  Pitts,  1943;  Grossberg,  1988,  Feldman  and  Ballard, 
1982;  Hertz  et  al.,  1990;  Lawrence,  1994,  Bishop  1995),  usually  called  "Neural  Network"  (NN),  is  a 
mathematical  or  computational  model  that  hies  to  simulate  the  structure  and/or  functional  aspects  of 
biological  neural  networks.  It  consists  of  an  interconnected  group  of  artificial  neurons  and  processes 
information  using  a  connectionist  approach  to  computation.  In  most  cases  an  ANN  is  an  adaptive  system 
that  changes  its  structure  based  on  external  or  internal  information  that  flows  through  the  network  during 
the  learning  phase.The  objective  of  the  neural  network  is  to  transform  the  inputs  to  the  network  into 
meaningful  outputs.  More  details  of  ANN  will  be  discussed  in  Section  4. 

ANN  has  distinct  features  over  other  approaches,  including: 

•  Self-learning  &  tuning  capability; 

•  Not  necessary  to  extract  the  thinking  patterns  of  experts; 

•  Robustness  towards  noisy  data,  thus  well  suited  for  sensorial  data  processing; 

•  Ability  to  implicitly  detect  complex  nonlinear  relationships  between  dependent  and  independent 
variables; 

•  Availability  of  multiple  training  algorithms;  and 

•  No  need  to  assume  an  underlying  data  distribution  such  as  usually  done  in  statistical  modelling. 

ANN,  like  other  methods,  also  has  some  limitations,  for  example: 

•  Requires  a  substantial  training  set; 

•  Black  box  nature  without  explicit  explanation; 

•  Greater  computational  burden; 

•  Unable  to  combine  numeric  data  with  linguistic/logic  information; 

•  Incapable  of  managing  imprecise  or  vague  information; 

•  Difficult  to  reach  global  minimum  even  by  complex  Back-Propagation  (BP)  learning;  and 

•  Rely  on  trial-and-errors  to  determine  hidden  layers  and  nodes. 
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ANN  has  also  been  used  in  numerous  application  areas  including  military  target  recognition  (Himes  & 
Inigo  1992;  Ratches  et  al.,1997;  Roth  1990;  Zhao  &  Bao,  1996),  wargame  threat/situation  assessment 
(Rushing  et  al.,  2004;  Madeira  et  al.,  2010),  pattern  classification  (Zhang,  2000;  Zhai  et  al,  2006;  Ibrahim 
et  al.  2009),  emotion  recognition  (Khashman,  2008),  prediction  (Ahmed  2005;  Gan  et  al.,  2005), 
clustering  (Liu  &  Zheng,  1992;  Sanchez  et  al.,  2006),  function  approximation  (  Zainuddin  &  Pauline, 
2008),  speech  recognition  (Al-Alaoui  et  al,  2008;  Lim  et  al.,  2000),  medicine  diagnosis  (Ozyilmaz  & 
Yildirim,  2003;  Ahmed,  2005;  He  et  al.2009),  computer  vision,  speech  recognition,  biometrics, 
handwriting  recognition,  portfolio  management,  financial  forecasting,  quality  control,  fraud  detection,  etc. 
In  Al-driven  wargame  replications,  ANN’s  appropriate  application  might  be  best  applied  at  the  situation  / 
pattern  recognition  level. 

3.1.4  Evolutionary  approaches 

Evolutionary  computation  is  a  subfield  of  computational  intelligence  that  involves  combinatorial 
optimization  problems  with  inspirations  drawn  from  biological  evolution.  Genetic  algorithms  are  the  most 
popular  type  of  evolutionary  computational  approach. 

Genetic  algorithms  (GA)  (Fraser  1957;  Barricelli  1957;  Holland,  1975;  Fogel,  1998,  2006)  are  a  search 
technique  used  in  computing  to  find  exact  or  approximate  solutions  to  optimization  and  search  problems. 
Genetic  algorithms  are  categorized  as  global  search  heuristics  with  the  mechanism  inspired  by 
evolutionary  biology  such  as  inheritance,  mutation,  selection,  and  crossover.  They  are  implemented  in  a 
computer  simulation  in  which  a  population  of  abstract  representations  of  candidate  solutions  to  an 
optimization  problem  evolves  toward  better  solutions. 

Advantages  of  genetic  algorithms  include 

•  Ability  to  quickly  scan  a  vast  solution  set; 

•  Capable  of  discarding  bad  proposals  without  affecting  the  end  solution  negatively;  and 

•  Not  necessary  to  know  domain  rules  and  easy  to  parallelize. 

Some  disadvantages  of  genetic  algorithms  are  as  follows: 

•  Possible  to  evolve  into  a  dead  end; 

•  Quality  as  good  as  the  evaluation  function  (often  hardest  part);  and 

•  No  guaranteed  convergence  even  to  local  minimum. 

Some  examples  of  GA’s  applications  include  situation  assessment  (Gonsalves  et  al.,  2000),  strategy 
generation  (Revello  and  McCartney,  2002),  wargame/gaming/strategy  selection  (Revello  &  McCartney, 
2002;  Pew  &  Mavor,  1998;  Watson  et  al.,  2010  ;  Brainz,  2010;  Periaux  et  al.,  2001;  Dworman  et  al., 
2010;  Marks,  2010),  planning  (Guitouni  and  Belfares,  2003;  Allaire  et  al.,  2009;  Kewley  &  Embrechts, 
2002;  Boukhtouta  et  al.,  2004;  Pew  and  Mavor,  1998),  trajectory  planning  of  robotics  (Davidor,  1991; 
Miryazdi  &  Khaloozadeh,  2002),  chemistry  &  biology  (Weber,  1998;  Padgett  &  Saad,  2009;  Leardi, 
2001),  clustering  (Fernandez  et  al.,  2010),  combination  optimization,  scheduling,  etc.  GA  may  be  used 
for  the  selection  of  strategy/tactics/courses  of  action  in  Al  wargame  replications. 
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3.1.5  Memory-based  approaches 


Memory-based  AI  approaches  consist  of  various  heuristic  search,  instance/example -based, 
case/experience-based,  and  analogical  methods.  This  section  reviews  case-based  reasoning,  a  typical 
method  in  the  memory-based  approaches. 

Case-based  reasoning  (CBR)  (Schank,  1982,  1990;  Riesbeck  &  Schank,  1989;  Kolodner,  1993),  broadly 
construed,  is  the  process  of  solving  new  problems  based  on  the  solutions  of  similar  past  problems.  An 
auto  mechanic  who  fixes  an  engine  by  recalling  another  car-  that  exhibited  similar  symptoms  is  using  case- 
based  reasoning.  Case-based  reasoning  is  a  prominent  kind  of  analogy  making. 

Case-based  reasoning  has  been  formalized  for  purposes  of  computer  reasoning  as  a  four-step  process:  (1) 
Retrieve :  given  a  target  problem,  retrieve  cases  from  memory  that  are  relevant  to  solving  it,  (2)  Reuse : 
map  the  solution  from  the  previous  case  to  the  target  problem,  (3)  Revise:  if  necessary,  revise  the  new 
solution,  and  (4)  Retain:  store  the  resulting  experience  as  a  new  case  in  memory. 

Compared  to  other  AI  methods,  CBR  has  some  apparent  advantages,  e.g., 

•  No  need  to  understand  the  domain  completely; 

•  Suitable  for  domains  without  precise  mathematical  or  algorithm  models; 

•  Naturally  reflecting  human’s  thinking  in  many  situations,  easy  knowledge  acquisition;  and 

•  Capable  of  learning  from  both  successes  and  failures. 

Some  limitations  of  CBR  contain: 

•  Hard  to  get  sufficient  cases; 

•  Hard  to  define  similarity  measurement  between  cases; 

•  Difficult  to  validate  the  systems;  and 

•  A  type  of  inductive  reasoning  without  guaranteed  generalization. 

CBR  has  also  been  used  in  various  application  areas,  for  example,  planning  and  decision  support  (Liao, 
2000;  Moriarty,  2000;  Lachevet,  2009;  Munoz -A vial  et  al,  1999;  Boukhtouta  et  al,  2004;  Pew  and  Mavor, 
1998;  Talbot,  2001),  situation  assessment  (Looney,  2003;  Pew  and  Mavor,  1998;  Gupta  and  Mukherjee, 
2009),  air  traffic  control  (Allendoerfer  and  Weber,  2004),  spatial  analysis  (Holt  and  Benwell,  1996), 
customer  services  (Acorn  and  Walden,  1992),  design  (Maher  et  al.,  1995),  clinical  problem  solving 
(Kolodner  &  Kolodner,  1987),  education  (Brown  et  al,  1989),  legal  reasoning,  medical  diagnosis, 
scheduling,  help-desk  support,  etc.  In  AI  wargame  replications,  CBR  may  be  used  for  solving  problems  in 
planning,  decision  making  and  problem  solving. 

3.1.6  Summary  of  Al  approaches 

The  following  Table  3  summarizes  these  AI  approaches,  providing  brief  descriptions,  advantages, 
disadvantages  and  main  applications. 
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Table  3:  Summary  of  AI  approaches 


Approach 

Category 

Approach 

Description 

Advantages 

Disadvantages 

Main  Applications 

Logic -based 

Decision  trees 

FOL  rules 
(rule -based) 

Fuzzy  logic 

Non¬ 

monotonic 

logic 

Using  decision  trees, 
propositional  / 
predicate  logic,  fuzzy 
logic  or  non¬ 
monotonic  logic  for 
reasoning 

•  Useful  for  decision  making 

•  Can  represent  human 
thinking  pattern 

•  Natural  knowledge 
representation 

•  Simple  to  understand 

•  Easy  to  get  explanation 

•  Require  experts’ 
knowledge  explicitly 
specified 

•  Knowledge  acquisition  is 
difficult 

•  Exhausted  search 

•  No  self-learning  / 
organization  ability 

•  Situation  assessment 

•  Decision  analysis 

•  Planning 

•  Expert  systems 

•  Learning:  pattern 
classification 

•  Prediction 

•  Control 

Probabilistic  / 

Kernel 

Naive  Bayes 

Bayesian 

Networks 

HMM 

SVM 

Use  Bayes  Theorem, 
and  /  or  acyclic 
directed  graph  as 
foundation  of 

inference. 

•  A  sound  decision  theory 

•  Consist,  theoretically  solid 
mechanism  for  processing 
uncertain  information 

•  Flexible  applicability 

•  A  clear  semantic 
interpretation  of  the  model 
parameters 

•  Allowing  different 
variables  types 

•  Handling  missing  data 

•  Does  not  support 
feedback  loop 

•  Quality  depending  on  the 
quality  of  the  prior 
beliefs  or  model 

•  Difficult  to  examine  the 

solutions 

•  Difficult  to  get  the 
probability  in  some 
domains. 

•  NP-hard  calculation 

costs 

•  Situation  /  threat 

assessment 

•  Decision  making 

•  Planning 

•  Prediction 

•  Pattern  recognition  and 
classification 

•  Knowledge  discovery 

•  Information  fusion 

•  Data  /text  mining 

Connectionism 

Neural 

Networks 

An  approach  that 
models  mental  or 

behavioural 
phenomena  as  the 

•  Not  necessary  to  extract  the 
thinking  patterns  of  domain 
experts 

•  Robustness  towards  noisy 

•  A  substantial  training  set 
is  required 

•  Black  box  nature 

•  Large  computational 

•  Threat/situation 

assessment 

•  Prediction 

•  Pattern  recognition 

emergent  processes  of 
interconnected 
networks  of  simple 
units  (neurons) 

data 

•  Ability  to  implicitly  detect 
complex  nonlinear 
relationships  between 
dependent  and  independent 
variables. 

burden. 

•  Target  recognition 

•  Emotion  recognition 

Evolutionary 

Genetic 

Algorithms 

A  popular  search- 
based  evolutionary 
algorithm  to  find  exact 
or  approximate 
solutions  to 
optimization  and 
search  problems. 

•  Can  quickly  scan  a  vast 
solution  set. 

•  Bad  proposals  do  not  affect 
the  end  solution  negatively 
as  they  are  simply 
discarded. 

•  Easy  to  parallelize 

•  Possible  to  evolve  into  a 

dead  end 

•  Quality  as  good  as 
evaluation  functions 

•  No  guaranteed 
convergence  even  to 

local  minimum. 

•  Planning 

•  Situation  assessment 

•  Strategy  generation 

•  Strategy  selection 

•  Optimal  route  selection 

•  Optimal  design 

Memory-based 

Case-based 

A  process  of  solving 
new  problems  based 
on  the  solutions  of 
similar  past  problems, 
including  four 
processes:  retrieve, 
reuse,  revise,  and 

retain. 

•  Can  be  used  in  problem 
domains  that  are  not  well 

understood. 

•  Can  re-use  previous 
successfully  solutions 

•  Relatively  easy  to  set  up  a 
knowledge  base. 

•  Adding  examples  are  easier 
than  rules. 

•  Can  learn  from  previous 
failed  examples 

•  Explanation  becomes 
easier. 

•  Defining  the  similarity 
between  cases  is 

difficult. 

•  Previous  experience  may 
not  be  validated. 

•  May  not  have  sufficient 
similar  cases. 

•  Validation  of  the  system 
is  difficult. 

•  Planning 

•  Decision  making 

•  Situation  assessment 

•  Air  Traffic  Control 

•  Analysis  and  design 

•  Problem  solving 

•  Scheduling 

3.2  Al  and  HBR  Tools 


3.2.1  HBR  tools 

There  are  many  tools  and  approaches  for  modelling  human  behaviour  /  performance,  such  as  the 
IPME  (Integrated  Performance  Modelling  Environment),  GOMS  (Goals,  Operators,  Methods  and 
Selection  rules);  EPIC  (Executive-Process  Interactive  Control),  HOS  (Human  Operator 
Simulator),  OMAR  (Operator  Model  Architecture),  etc.  This  section  focuses  mainly  on  the  HBR 
tools  with  obvious  AI  reasoning  approaches  and  functions,  including  the  Adaptive  Control  of 
Thought  -  Rational  (ACT-R),  Belief-Desire-Intention  (BDI),  Connectionism  Learning  with 
Adaptive  Rule  Induction  On-line  (CLARION),  Man  Machine  Integrated  Design  and  Analysis 
System  (MIDAS),  Situation  Awareness  Model  for  Pilot-in-the-Loop  (SAMPLE)  and  State, 
Operator  And  Result  (Soar). 

3.2.1 .1  Adaptive  Control  of  Thought  -  Rational  (ACT-R) 

ACT-R  is  a  cognitive  architecture  mainly  developed  by  John  Robert  Anderson  at  Carnegie 
Mellon  University  (Anderson,  1990;  Anderson  &  Lebiere,  1998;  Anderson  et  al.,  2004).  Like  any 
other  cognitive  architecture,  ACT-R  aims  to  define  the  basic  and  irreducible  cognitive  and 
perceptual  operations  that  enable  the  human  mind.  In  theory,  each  task  that  humans  can  perform 
should  consist  of  a  series  of  these  discrete  operations. 

Figure  7  shows  the  architecture  of  ACT-R,  consisting  of  the  following  components: 

•  Modules:  including  perceptual-motor  modules,  and  memory  modules:  declarative 
memory  for  facts  and  procedural  memory  for  IF-THEN  rules. 

•  Buffers:  ACT-R  accesses  its  modules  (except  for  the  procedural-memory  module) 
through  buffers.  For  each  module,  a  dedicated  buffer  serves  as  the  interface  with  that 
module. 

•  Pattern  Matcher.  The  pattern  matcher  searches  for  a  production  that  matches  the  current 
state  of  the  buffers. 
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Figure  7:  Architecture  of  ACT -R  (Anderson  et  al.,  2004;  Chong  et  ah,  2007) 

A  fundamental  characteristic  of  ACT-R  is  that  it  is  based  on  a  production  system  theory.  The 
basic  premise  of  a  production  system  theory  is  that  a  cognitive  skill  is  composed  of  conditional 
statements  known  as  production  rules.  A  production  rule  is  a  statement  that  describes  an  action 
which  would  be  taken  if  a  condition  is  met,  sometimes  referred  to  as  a  condition-action  pair. 

The  main  advantages  of  ACT-R  include  a  powerful  reasoning  method,  very  active  user 
community  (primarily  psychologists  and  cognitive  scientists),  strong  training,  support,  and 
commitment  to  “user  friendly”  software,  and  is  the  most  well  known  of  current  psychology-based 
systems. 

ACT-R  also  has  some  limitations,  for  instance,  it  is  primarily  an  academic  research  system,  its 
value  for  large  applied  problems  has  not  been  clearly  demonstrated,  it  has  a  debated  scientific 
basis,  the  perceptual/motor  representations  are  lagging,  and  there  are  many  parameters  affecting 
performance. 

Most  applications  of  ACT-R  focus  on  cognitive  experiments,  including  human -computer 
interaction,  intelligent  tutoring  systems,  experiments  for  behaviour  moderators,  experiments  of 
military  modelling  and  simulation,  learning  and  memory,  problem  solving  and  decision  making, 
language  and  communication,  perception  and  attention,  cognitive  development,  and  individual 
differences. 

3.2. 1.2  Belief-Desire-Intention  (BDI) 

BDI  is  based  on  the  Dennett’s  theory  of  intentional  systems  (Dennett,  1987)  and  the  theory  of 
human  practical  reasoning  (Bratman  et  al.,  1988).  Originally  developed  as  a  system  that  can 
reason  and  plan  in  a  dynamic  environment,  BDI  meets  real-time  constraints  by  reducing  the  time 
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used  in  planning  and  reasoning.  BDI  is  designed  to  be  situated,  goal  directed,  reactive,  and  social. 
This  means  that  a  BDI  agent  is  able  to  react  to  changes  and  communicate  in  their  embedded 
environment  as  it  attempts  to  achieve  its  goals.  Mechanisms  for  responding  to  new  situations  or 
goals  during  plan  formation  for  general  problem  solving  and  reasoning  in  real  time  processes  are 
also  included  in  most  BDI  systems  (Georgeff  and  Ingrand,  1989;  Sardina  et  al.,  2006). 

BDI  architecture  consists  of  Beliefs,  Desires  and  Intentions.  Beliefs  represent  the  information 
state  of  the  agent.  Desires  represent  the  motivational  state  of  the  agent.  And  Intentions  represent 
the  deliberative  state  of  the  agent  -  what  the  agent  has  chosen  to  do. 

There  are  some  limitations  and  criticisms  of  BDI,  including  doubt  surrounding  the  sufficiency  of 
the  three  attributes  (Rao  and  Georgeff,  1955),  that  it  is  not  adaptable  for  learning  behaviour, 
agents  have  no  ability  to  interact  with  other  agents,  there  is  no  explicit  goal  representation,  no 
forward  planning  capability,  and  that  the  multi-modal  logic  underlining  BDI  has  little  relevance 
in  practice  (Rao  &  Georgeff,  1995). 

Some  implementations  of  BDI  consist  of  Procedural  Reasoning  System  (PRS)  (Georgeff  & 
Ingrand,  1989;  Guerra-Hernandez  et  al.,  2004;  Sardina  et  al.,  2006),  Distributed  Multi-Agent 
Reasoning  System  (dMARS)  (d’lnverno  et  al.,  2004),  JACK  Intelligent  Agents  (Howden  et  al., 
2001),  and  CoJACK  (Evertsz,  et  al.,  2008)  -  an  extension  to  the  JACK  platform  that  adds  a 
cognitive  architecture  to  the  agents  for  eliciting  more  realistic  behaviours  in  virtual  environments. 

3. 2. 1.3  Connectionist  Learning  with  Adaptive  Rule  Induction  On-line 
(CLARION) 

CLARION,  with  its  root  in  neural  networks,  is  a  hybrid  architecture  that  incorporates  models  of 
both  implicit  and  explicit  memory  for  reasoning  and  learning  (Sun  et  al.,  2001).  Procedural 
knowledge,  i.e.  implicit  memory,  can  be  gradually  accumulated  with  repeated  practice,  and 
subsequently  applied  to  practiced  situations  with  minor  variations  (Chong  et  al.,  2007).  To  deal 
with  novel  situations,  declarative  knowledge  is  required  to  assist  in  the  exploration  of  new 
situations,  thereby  reducing  time  for  developing  specific  skills.  It  also  unifies  neural, 
reinforcement  and  symbolic  methods  to  perform  on-line,  and  bottom-up  learning.  Hence, 
CLARION  is  able  to  react  in  a  dynamically  changing  environment  without  any  pre-existing 
knowledge  installed  into  the  architecture  (Sun  and  Peterson,  1996,  1998). 

CLARON  is  an  integrative  architecture,  consisting  of  a  number  of  distinct  subsystems,  with  a 
dual  representational  structure  in  each  subsystem.  Its  subsystems  include  the  action-centered 
subsystem  for  action  control,  the  non-action-centered  subsystem  for  general  knowledge,  the 
motivational  subsystem  for  motivations  of  perception,  action  and  cognition,  and  the  meta- 
cognitive  subsystem  for  controlling  all  other  subsystems. 

CLARION  has  been  used  to  implement  intelligent  systems  (Sun  et  al.  2010).  Some  application 
examples  include  the  serial  reaction  time  task,  the  artificial  grammar  learning  task,  the  process 
control  task,  a  categorical  inference  task,  an  alphabetical  arithmetic  task,  and  the  Tower  of  Hanoi 
task. 
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3.2. 1.4  Man  Machine  Integrated  Design  and  Analysis  System  (MIDAS) 


MIDAS  is  a  system  for  simulating  one  or  more  human  operators  in  a  simulated  world  of  terrain, 
vehicles,  and  other  systems  (Laughery  and  Corker,  1997;  Pew  and  Mavor,  1998;  Tyler  et  al., 
1998).  The  primary  purpose  of  MIDAS  is  to  evaluate  proposed  human-machine  system  designs 
and  to  serve  as  a  test  bed  for  behavioural  modelling. 

The  overall  architecture  of  MIDAS  comprises  a  user  interface,  an  anthropometric  model  of  the 
human  operator,  symbolic  operator  models,  and  a  world  model.  The  user  interface  consists  of  an 
input  side  (an  interactive  GUI,  a  cockpit  design  editor,  and  equipment  editor,  a  vehicle  editor,  and 
an  activity  editor)  and  an  outside  (display  animation  software,  run-time  data  graphical  displays, 
summary  data  graphical  displays,  and  3D  graphical  displays).  The  human  operator  representation 
consists  of  physical  representation,  perception  and  attention,  updatable  world  representation, 
activity  representation,  scheduler  and  user  interfaces. 

MIDAS  uses  IF-THEN  rules  to  provide  flexible  problem-solving  capabilities.  Description 
activities  select  from  among  alternatives.  Six  generalized  decision  algorithms  are  available: 
weighted  additive,  equal  weighted  additive,  lexicographic,  elimination  by  aspect,  satisfying 
conjunctive,  and  majority  of  confirming  decisions. 

The  advantages  of  MIDAS  include  submodels  based  on  current  psychological  and  psychomotor 
theory  and  data,  task  loading  modelling  consistent  with  multiple  resource  theories,  potential  for 
military  simulations,  and  a  good  base  for  a  human  behaviour  representation.  Its  limitations  consist 
of  insufficient  behaviour  models,  is  too  cumbersome  for  most  military  simulations,  and  is  very 
labour-intensive  for  application  development. 

MIDAS  has  been  used  for  situation  awareness  (Shively  et  al.,  1997;  Burdick  &  Shively,  2000), 
and  operator  cognition  &  performance  (Gore  et  al.,  2009;  Boring  et  al.,  2008;  Gore  &  Jarvis, 
2005). 

3.2. 1.5  Situation  Awareness  Model  for  Pilot-in-the-Loop  Evaluation  (SAMPLE) 

SAMPLE  is  used  to  represent  individual  operators,  as  well  as  crews  of  complex  human  machine 
systems  (Baron  et  al.,  1980;  Zacharias  et  al.,  1981,  1994,  1996). 

The  SAMPLE  architecture  provides  a  general  framework  for  constructing  models  of  operators  of 
complex  systems,  particularly  in  cases  in  which  the  operators  are  engaged  in  information 
processing  and  controls  tasks.  SAMPLE  draws  heavily  on  modern  control  theory,  which  has 
enjoyed  considerable  success  in  the  modelling  of  human  control  behaviour.  The  belief-net  at  the 
core  of  the  situation  assessor  of  later  variants  appears  to  have  considerable  potential  for 
representing  situation  awareness.  However,  procedure  development  for  SAMPLE  models  would 
appear  to  be  quite  labour-intensive  since  there  seems  to  be  no  high-level  procedure  representation 
language. 

The  SAMPLE  architecture  consists  of  a  system  model  and  one  or  more  human  operator  models. 
The  system  model  takes  in  system  dynamics,  e.g.  ownship,  the  plant,  or  a  target,  which  is 
modeled  by  partial  differential  equations  of  motion  (e.g.,  point  mass  equations  for  vehicle 
trajectory).  The  system  dynamics  can  be  modeled  at  any  level  of  complexity  desired. 
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A  human  operator  model  exists  for  each  crew  member.  It  consists  of  sensory  and  effector 
channels  and  several  processors:  an  information  processor  monitoring  the  system,  and  situation 
assessors  generating  random  situation  and  task  procedures.  The  sensory  channels  model  visual 
and  auditory  sensing.  Both  are  based  on  an  optimal  control  model  with  no  perceptual  delay. 

SAMPLE  has  been  used  in  the  examination  of  crew  procedures,  for  example,  the  evaluation  of  air 
traffic  alerting  systems  in  a  free  flight  environment. 

3.2. 1.6  State,  Operator  And  Result  (Soar) 

Soar  is  another  symbolic  cognitive  architecture  (as  shown  in  Figure  8)  created  by  John  Laird, 
Allen  Newell  and  Paul  Rosenbloom  at  Carnegie  Mellon  University  (Laird  et  al.,  1987;  Newell, 
1990;  Lehman  et  al.,  2010). 

The  main  goal  of  the  Soar  architecture  is  to  create  a  system  that  is  able  manage  the  full  range  of 
capabilities  of  an  intelligent  agent,  from  highly  routine  to  extremely  difficult  open-ended 
problems.  According  to  the  view  underlying  Soar,  such  architecture  must  be  able  to  create 
representations  and  use  appropriate  forms  of  knowledge  (such  as  procedural,  declarative, 
episodic,  and  possibly  iconic). 

Soar  is  based  on  a  production  system,  i.e.  it  uses  explicit  production  rules  to  govern  its  behavior 
(these  are  roughly  of  the  form  "if...  then...",  as  also  used  in  expert  systems).  Problem  solving  can 
be  roughly  described  as  a  search  through  a  problem  space  (the  collection  of  different  states  which 
can  be  reached  by  the  system  at  a  particular  time)  for  a  goal  state  (which  represents  the  solution 
for  the  problem).  This  is  implemented  by  searching  for  the  states  which  bring  the  system 
gradually  closer  to  its  goal.  Each  move  consists  of  a  decision  cycle  which  has  an  elaboration 
phase  (in  which  a  variety  of  different  pieces  of  knowledge  bearing  the  problem  are  brought  to 
Soar's  working  memory)  and  a  decision  procedure  (which  weighs  what  was  found  on  the  previous 
phase  and  assigns  preferences  to  ultimately  decide  the  action  to  be  taken). 


Figure  8:  Soar  architecture  (Chong  et  al.,  2007;  Lehman  et  al.,  2010). 
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Soar  is  best  described  at  two  levels,  called  the  problem  space  level  and  the  symbol  level.  At  the 
problem  space  level,  Soar  casts  all  cognitive  activity  as  transformations  of  states  by  operators 
within  a  state  space.  In  a  state  space,  there  is  a  single  current  state,  which  encodes  working 
information  about  the  problem  or  situation  being  processed.  Cognitive  processing  occurs  by 
means  of  operators,  which  apply  to  the  current  state  to  yield  a  new  current  state.  In 
straightforward  cases,  processing  consists  of  a  repeated  cycle  in  which  Soar  first  picks  an 
operator,  and  then  applies  it  to  transform  the  current  state  into  a  new  one. 

Advantages  of  Soar  (Kieras,  2010)  include  a  well  established  in  AI  community,  good  software 
support  and  training,  commercial-grade  applications  developed  and  sold,  freely  available,  and 
definite  track  record  of  success  on  large  problems  (thousands  of  rules). 

Soar  also  has  some  limitations,  for  example,  psychological  basis  of  learning  and  reasoning  are  not 
as  well  developed  as  others,  and  reputed  to  be  difficult  to  program,  even  with  excellent  software 
support.  There  are  many  applications  with  Soar  in  various  application  areas,  in  particular  in 
military  modelling  and  simulations  (Pew  &  Mavor,  1998);  for  example,  the  Synthetic  Theater  of 
War  (STOW-97),  Tactical  Air-Soar  (TacAir-Soar,  JCATS,  Rotary-Wing  Aircraft-Soar  (RWA- 
Soar),  Fixed-Wing  Aircraft-Soar  (FWA-Soar),  Special  Operation  Forces-Soar  (  SOFSoar),  etc. 

3.2.1 .7  Summary  of  HBR  tools 

The  following  Table  4  summarizes  these  FIBR  tools. 
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Table  4:  Summary  ofHBR  tools. 


Tool 

Original 

Purpose 

Memory  /  KR 

Functions 

Validating 

Notes 

ACT-R 

Model  problem 
solving  and 
learning 

Network  or 

schema-like 
structures  plus 
productions 

•  Learning:  Weight  adjustment  learning 

•  Planning:  Creating  new  Planning 

•  Decision  making:  knowledge-based, 

Bayesian 

•  Situation  Awareness:  Overt  and  inferred 

Extensive 

at  many 

levels 

Focusing  on  single,  specific 
information  processing  tasks;  has 
not  yet  been  scaled  up  to  complex 
multitasking  situations  or  high- 
knowledge  domains. 

BDI 

Problem  solving 

Object 

oriented 

•  Learning:  no 

•  Planning:  instantiates  general  plans 

•  Decision  making:  BDI-based 

•  Situation  awareness:  overt 

n/a 

Based  on  three  attributes:  Belief, 

Desire,  and  Intention. 

CLARION 

Extending 
neural  networks 

functions 

Neural 

networks  and 

production 

rules 

•  Learning:  neural  networks -based 

•  Planning:  instantiates  general  plans 

•  Decision  making:  rule-based 

•  Situation  awareness:  neural-network-based 

n/a 

Combining  neural  networks  and 
production  rules. 

MIDAS 

Evaluate 

interfaces  and 
procedures 

Frames,  rules 

•  No  learning 

•  Planning:  instantiates  general  plans 

•  Decision  making:  knowledge-based, 

Bayesian 

•  Situation  awareness:  overt 

Full  model 

Scripted  behaviors. 

SAMPLE 

Evaluate  crew 

procedures, 

equipment 

Objects, 

Production 

rules 

•  Learning:  no 

•  Planning:  Instantiates  general  plans 

•  Decision  Making:  Knowledge-based, 

Bayesian 

Control 

tasks 

(OCM) 

Has  been  used  in  small-scale 
military  simulations. 

K> 

VO 


o 


•  Situation  Awareness:  overt 

Soar 

Model  problem 
solving  and 
learning. 

Productions 

•  Learning:  learning  by  chunking 

•  Planning:  can  create  new  plans 

•  Decision  making:  knowledge-based 

•  Situation  awareness:  overt  and  inferred 

Extensive 
at  multiple 
levels 

Has  been  used  in  military 
simulations:  e.g.,  synthetic  theater 
of  war-Europe  [STOW-E],  STOW- 
97,  etc. 

3.2.2 


Other  Al  tools 


3.2. 2.1  Language  of  Agents  for  Modelling  Performance  (LAMP) 

Language  of  Agents  for  Modelling  Performance  (LAMP)  (Guo  et  al.,  2005a,  2005b)  is  a 
knowledge  representation  and  reasoning  approach  to  provide  intelligent  /  human-like  behaviour 
for  HBR  in  modelling  and  simulation  tools,  such  as  Integrated  Performance  Modeling 
Environment  (IPME)  or  OneSAF.  It  is  able  to  represent  both  deterministic  and  uncertain  (Guo  et 
al.,  2008)  knowledge,  as  well  as  supporting  semantic-similarity-based  associative  reasoning,  and 
rule-based  abstract  deduction. 

Figure  9  is  a  conceptualization  of  LAMP,  encompassing  an  Interactivelnterface  to  human 
behaviour  representation  tools  and  an  Aspect  Network.  The  Interactivelnterface  module  receives 
the  data  from  M&S  tools,  activates  the  LAMP  engines  to  get  solutions,  and  sends  the  results  to 
M&S  tools  for  the  next  phase  of  simulation. 


M&S  Tools 
IPME,  OneSAF,  etc. 


Figure  9:  LAMP  component  organization. 

The  core  of  LAMP  is  an  Aspect  Network,  in  which  each  Aspect  is  a  knowledge  unit  with  Goals, 
Sensinglnterface,  Dual-Process  Knowledge,  Motorlnterface,  and  Sociallnterface.  During 
simulation,  M&S  tools  communicate  with  LAMP’S  Interactivelnterface  to  activate  the  dual¬ 
process  reasoning  sub-system,  send  data  to  and  get  conclusions  from  the  reasoning  sub-system, 
and  then  apply  the  reasoning  results  for  the  next  step  of  task  simulation. 
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Goals  in  an  Aspect  describe  the  solutions  or  conclusions  that  can  be  achieved  by  this  Aspect. 
Sensinglnterface  receives  the  data  from  the  M&S  tool  and  converts  the  data  into  an  internal 
representation.  Motorlnterface  transmits  the  reasoning  results  to  the  simulated  tasks  in  an 
appropriate  format.  Sociallnterface  provides  mechanisms  to  interact  with  other  Aspects  for 
collaborated  resolutions  or  reasoning. 

The  kernel  of  Aspect  is  the  Dual-Process  Knowledge  that  comprises  modules  for  Associative 
Reasoning,  Abstract  Reasoning,  Approach  Engines,  Adaptive  Interface  and  Meta  Reasoning. 
LAMP  integrates  dual-process  and  behaviour  moderators  (Halford  et  al.,  2006;  Evans,  2003, 
Sloman,  1996,  2002)  to  enhance  reality. 

LAMP  has  been  used  as  the  reasoning  engine  in  the  Simulated  Operators  for  Networks  (SIMON) 
project  in  DRDC  Toronto.  Examples  of  applications  include  battlefield  reasoning  with  personality 
effects,  fuzzy  logic  helicopter  flight  control,  and  analogical  problem  solving  in  search  and  rescue. 

Compared  with  other  reasoning  systems,  LAMP’S  main  features  include  the  integration  of  human 
dual-process  thinking,  multiple  approaches,  human  behaviour  moderators,  human  memory  and 
analogical  models. 

3.2.2.2  Logic-based  software 

Two  general  AI  programming  languages  including  List  Programming  (Lisp)  (McCarthy,  1960) 
and  Programming  in  Logic  (Prolog)  (Warren,  1977)  are  proposed  for  generic  AI  programming. 
The  following  provides  a  list  of  commercial  and  free  available  implementations. 

Lisp  is  the  earliest  AI  programming  language.  Today,  the  most  widely  known  general-purpose 
Lisp  dialects  are  Common  Lisp  and  Scheme.  Commercial  implementations  of  Lisp  include 
(online  information  available  on  February  5,  2010): 

•  Allegro  Common  List,  http://www.franz.com/products/allegrocl/ 

•  Corman  Common  Lisp,  http://www.cormanlisp.com/ 

•  LispWorks,  http://www.lispworks.com/ 

•  Scieneer  Common  Lisp,  http://www.scieneer.com/scl/. 

Some  freely  redistributable  implementations  of  Lisp  consist  of: 

•  Armed  Bear  common  Lisp,  http://common-lisp.net/project/armedbear/ 

•  CLISP,  http://clisp.cons.org/ 

•  Clozure  CL,  http://trac.clozure.com/ccl;  CMUCL  at  http://www.cons.org/cmucl/ 

•  Embeddable  Common  Lisp  (ECL)  ,  http://ecls.sourceforge.net/ 

•  GNU  Common  Lisp,  http://www.gnu.org/software/gcl/ 

•  Macintosh  Common  Lisp,  http://www.digitool.com/ 

•  Movitz,  http://common-lisp.net/project/movitz/ 

•  Poplog,  http://www.poplog.org/ 
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Steel  Bank  Common  Lisp,  http://sbcl.sourceforge.net/. 


Prolog  (Warrant.  1977)  is  a  general  purpose  logic  programming  language  associated  with 
artificial  intelligence  and  computational  linguistics.  Commercial  software  examples  of  Prolog 
include  (online  information  available  on  February,  5,  2010): 

•  Amzi  Prolog,  http://www.amzi.com/ 

•  Arity  Prolog,  http://www.arity.com/?Tab=products&tab2=prolog 

•  BinProlog,  http://www.binnetcorp.com/BinProlog/ 

•  IF/Prolog,  http://www.ifcomputer.com/IFProlog/ 

•  LPA-Win,  http://www.lpa.co.uk/ind_pro.htm 

•  MINERVA  Prolog,  http://www.ifcomputer.com/MINERVA/ 

•  Quintus  Prolog:  http://www.sics.se/quintus/ 

•  SICStus:  http://www.sics.se/isFsicstuswww/site/order.html 

•  Trinc-Prolog:  http://www.trinc-prolog.com/ 

•  Visual  Prolog:  http://www.visual-prolog.com/. 

Examples  of  freely  or  shareware  Prolog  implementations  comprise  (online  information  available 
on  February,  5,  2010): 

•  Aquarius  Prolog:  http://www.info.ucl.ac.be/~pvr/aquarius.html 

•  B-Prolog:  http://www.probp.com/ 

•  BinProlog:  http://www.binnetcoip.com/BinProlog/ 

•  CIAO  Prolog:  http://www.ciaohome.org/ 

•  Cu_Prolog:  http://www.freebsdsoftware.org/lang/cu-prolog.html 

•  EZY-Prolog:  http://www.ezy-software.com/ 

•  GNU  Prolog  at  http://www.gprolog.org/ 

•  Open  Prolog:  http://www.scss.tcd.ie/misc/open-prolog/ 

•  Poplog:  http://www.cs.bham.ac.uk/research/projects/poplog/freepoplog.html 

•  Strawberry  Prolog:  http://www.dobrev.com/ 

•  SWI-Prolog:  http://www.swi-prolog.org/ 

•  Visual  Prolog  Personal  Edition:  http://www.visual-prolog.com/. 

In  addition,  there  are  also  several  rule-based  tools  available,  e.g.,  the  C  Language  Integrated 
Production  System  (CLIPS)  (Girratano  and  Riley,  1998)  at 
http://www.siliconvalleyone.com/clips.htm,  ACT-R  at  http://act-r.psy.cmu.edu/actr6/  and  Soar  at 
http://sitemaker.umich.edu/soar/soar_software_downloads.  (Note:  Online  information  available 
on  February  5,  2010.) 


DRDC  CORA  CR  2010-269 


33 


3.2. 2. 3  Bayesian  networks  software 

The  following  list  contains  some  commercial  software  of  Bayesian  belief  networks  as  follows 
(online  information  available  on  February  2,  2010): 

•  AgenaRisk  Bayesian  network  tool,  http://www.agenarisk.com 

•  BayesBuilder,  http://www.snn.ru. nl/nijmegen/index.php3?page=3 1 

•  Bayesia,  http://www.bayesia.com 

•  Bayesian  network  application  library,  http://www.norsys.com/netlibrary/index.htm 

•  BNet,  http://www.cra.com/bnet 

•  Causeway,  http://www.inet.saic.com/ 

•  DBL  Interactive,  http://www.decisionbasedlearning.org/ 

•  Dezide,  http://www.dezide.com 

•  dVelox  ,  http://aparasw.com/dVelox 

•  Hugin,  http://www.hugin.com 

•  MSBNx:  BN  from  Microsoft  Research,  http://research.microsoft.com/adapt/MSBNx/ 

•  Netica,  http://www.norsys.com 

•  Promedas  Bayesian  medical  decision  support,  http://www.promedas.nl 

•  ProBayes,  http://www.probayes.com 

•  Quiddity,  http://www.iet.webfactional.com/quiddity.html. 

There  is  a  great  number  of  free  and  open  source  software  of  Bayesian  belief  networks  (online 
information  available  on  February  2,  2010): 

•  Ace,  http://reasoning.cs.ucla.edu/ace 

•  Alspace,  http://aispace.org/bayes 

•  BANJO:  BN  in  Java,  http://www.cs.duke.edu/~amink/software/banjo 

•  BANSY3,  http://www.dynamics.unam.edu/DinamicaNoLineal3/bansy3.htm 

•  Bayesian  Logistic  Regression  Software,  http://stat.rutgers.edu/~madigan/BBR/ 

•  BNJ:  Bayesian  Network  tools  in  Java,  http://bnj.sourceforge.net/ 

•  BN4R,  http://bn4r.rubyforge.org/ 

•  BNJ:  BN  in  Java,  http://bnj.sourceforge.net/ 

•  BN  PowerPredictor:  BN  based  classifier  learning, 
http://webdocs.cs.ualberta.ca/~jcheng/bnsoft.htm 

•  BNT:  BN  Toolbox  for  MatLab,  http://bnt.sourceforge.net/ 

•  dlib  C++  Library,  http://dclib.sourceforge.net/ 
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•  FDEP,  http://www.cs.bris.ac.uk/~flach/fdep/ 

•  GeNIe  &  SMILE:  BN  for  Windows,  http://genie.sis.pitt.edu/ 

•  JavaBayes:  BN  in  Java,  http://www.cs.cmu.edu/~javabayes/Home/  or 
http://www.pmr.poli.usp.br/ltd/Software/javabayes/ 

•  jBNC:  BN  classifier  in  Java  ,  http://jbnc.sourceforge.net/ 

•  JNCC2,  extension  of  Naive  Bayes  Classifier  in  Java), 
http://www.idsia.ch/~giorgio/jncc2.html 

•  Mocapy:  BN  in  Python,  http://mac.softpedia.com/get/Math-Scientific/Mocapy.shtml 

•  MSBNx:  Microsoft  Bayesian  Network  Editor,  http://research.microsoft.com/en- 
us/um/redmond/groups/adapt/msbnx/ 

•  OpenBayes,  http://www.openbayes.org 

•  pebl:  BN  in  Python,  http://pebl-project.googlecode.com 

•  ProBT-  free  version  of  the  ProBAYES',  http://www.probayes.com 

•  PNL,  http://sourceforge.net/projects/openpnl/ 

•  Pulcinella,  tool  in  Common  Lisp,  http://iridia.ulb.ac.be/pulcinella/Welcome.html 

•  RISO,  http://sourceforge.net/projects/riso/ 

•  Samlam,  http://reasoning.cs.ucla.edu/samiam  (See  also  Ace,  above) 

•  UnBBayes,  http//unbbayes. sourceforge.net/. 

3. 2. 2.4  Neural  networks  software 

Some  available  commercial  software  of  Artificial  Neural  Networks  includes  (online  information 
available  on  February  2,  2010): 

•  Alyuda  Neurointelligence,  http://www.alyuda.com/neural-networks-software.htm. 

•  BioComp  iModel(tm),  http://www.biocompsystems.com/products/imodel/ 

•  COGNOS  4Thought,  http://www-01.ibm.com/software/data/cognos/ 

•  BrainMaker,  http://www.calsci.com/ 

•  KnowledgeMiner,  http://www.knowledgeminer.com/ 

•  MATLAB  Neural  Net  Toolbox,  http://www.mathworks.com/products/neuralnet/ 

•  MemBrain,  http://www.membrain-nn.de/ 

•  NeuroSolutions,  http://www.nd.com/ 

•  NeuroXL,  NN  in  Excel,  http://www.neuroxl.com/.(NeuroDimension,  2010) 

•  NeuralWorks  Predict  3.0  and  Professional  I  I/PLUS, 
http://www.neuralware.com/index.jsp 

•  SPSS  Neural  Connection  2,  http://www.spss. com/press/template_view.cfm?PR_ID=165 
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•  STATISTICA,  http://www.statsoft.com/products/statistica-automated-neural-networks/ 

•  Synapse,  http://www.peltarion.com/products/synapse/  (Peltarion,  2010) 

•  Tiberius,  http://www.tiberius.biz/. 

There  are  also  some  free  software  and  shareware  of  neural  networks  (online  information  available 
on  February  2,  2010): 

•  NuClass7,  http://www-ee.uta.edu/eeweb/IP/Software/Software.htm 

•  Sciengy  RPF(tm),  http://sciengy.kuzmenko.net/ 

•  Sharky  Neural  Network,  http://sharktime.com/us_SharkyNeuralNetwork.html 

•  Stuttgart  Neural  Network  Simulator  (SNNS),  JavaNNS  (Stuttgart-University,  2010), 
http://www.ra.cs.uni-tuebingen.de/SNNS/ 

•  Emergent  (formerly  PDP++)  (Aisa  et  al.,  2008;  O’Reilly  et  al.,  2000), 
http://grey.colorado.edu/emergent/index.php/Main_Page 

•  Neural  Lab  (Guanajuato,  2010) , 
http://www.dicis.ugto.mx/profesores/sledesma/documentos/index.htm 

•  Neuro  Laboratory,  http://www.scientific-soft.com/7content/products/neurolab/main.htm 
(Scientific  Soft  2010). 

3.2.2.5  Description  logics  software 

•  Protege,  Protege-OWL,  SWOOP 

•  Description  logic  reasoners  -  PaCT++,  Pellet,  Racer-Pro,  Sim-DL 
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4  Development  roadmap 


This  section  examines  some  relevant  results  from  human  science,  including  neural  networks, 
human  memory  models,  analogy,  dual-process  thinking,  and  human  behaviour  moderators.  Based 
on  the  analysis  of  problems,  an  integrated  and  adaptive  AI  replication  engine  is  proposed  for 
modelling  human  interactors’  intentions  in  wargame  replications. 


4.1  Overview  of  human  behaviour  representation 

An  HBR  is  a  computer-based  model  that  mimics  either  the  behaviour  of  a  single  human  or  the 
collective  action  of  a  team  of  humans  (Pew  and  Mavor,  1998).  Figure  10  shows  an  integrated 
architecture  for  an  FIBR. 


Figure  10:  An  integrated  architecture  of  human  behaviour  representation  (Pew  &  Mavor,  1998). 

The  “Sensing  and  Perception”  module  in  the  architecture  acquires  the  environment  data  via 
models  of  vision,  hearing  or  perception,  and  then  sends  the  data  to  a  working  memory  store. 
There  are  considered  to  be  two  representations  of  human  memory:  working  memory  and  long¬ 
term  memory.  Working  memory  maintains  a  short-term  representation  of  data  from  the 
environment  and  interacts  with  long-term  memory  during  retrieval,  storage,  and  comparative 
activities.  There  are  three  kinds  of  components  in  long-term  memory:  declarative  knowledge, 
procedural  knowledge  and  various  tasks.  In  the  next  step,  working  memory  interacts  with  the 
cognitive  functions  associated  with  learning,  decision  making,  situation  awareness,  planning  and 
multitasking.  The  outputs  of  the  models  of  cognition  are  transmitted  to  “Motor  Behaviour”  to 
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model  the  reactions.  Finally,  the  status  of  the  environment,  e.g.  battlefield,  changes  and  another 
cycle  begins  again. 

There  are  many  FIBR  prototypes  proposed,  for  example,  Adaptive  Control  of  Thought  (ACT-R  ) 
(Anderson,  1990;  Anderson  et  al., 2004),  State,  Operator  And  Result  (Soar)  (Laird  et  al.,  1987; 
Newell,  1990),  COGnition  as  a  Network  of  Tasks  (COGNET)  (Zachary  et  al.,  1991;  Zachary  et 
al.,  1996),  Executive-Process  Interactive  Control  (EPIC)  (Meyer  and  Kieras,  1997a,  1997b), 
Human  Operator  Simulator  (HOS)  (Glenn  et  al.,  1992),  Integrated  Performance  Modeling 
Environment  (IPME)  (MAAD,  2010),  Man  Machine  Integrated  Design  and  Analysis  System 
(MIDAS)  (Laughery  and  Corker,  1997;  Banda  et  al.,  1991),  Operator  Model  Architecture 
(OMAR)  (Deutsch  and  Adams,  1995;  MacMillan  et  al.,  1997),  Situation  Awareness  Model  for 
Pilot-in-the-Loop  Evaluation  (SAMPLE)  (Baron  et  al.,  1980,  Zacharias  et  al.,  1981,  1994,  1996). 
A  high  level  review  of  these  systems  can  be  found  in  Pew  and  Mavor’s  book  (Pew  and  Mavor, 
1998). 

HBR  tools  have  made  great  contributions  to  current  modelling  and  simulation  initiatives,  but 
there  are  some  limitations.  For  example,  as  described  by  Pew  and  Mavor  (1998):  current  decision 
making  tools  are  rigid,  predictable,  not  flexible,  not  human-like,  not  variable  and  not  adaptable 
(Pew  and  Mavor,  1998). 

4.2  Related  results  from  Human  Science 

This  section  reviews  some  results  from  human  science  that  are  significant  factors  to  consider  for 
future  HBR  methodology  and  models,  including  neural  networks,  human  memory  models, 
analogy,  dual-process  thinking  and  behaviour  moderators. 

4.2.1  Neural  networks 

In  neuroscience,  a  neural  network  (Barr  and  Kiernan,  1988)  describes  a  population  of  physically 
interconnected  neurons  or  a  group  of  disparate  neurons  whose  inputs  or  signaling  targets  define  a 
recognizable  circuit  (Figure  11).  Communication  between  neurons  involves  an  electrochemical 
process.  The  interface  through  which  they  interact  with  surrounding  neurons  usually  consists  of 
several  dendrites  (input  connections),  which  are  connected  via  synapses  to  other  neurons,  and  one 
axon  (output  connection).  If  the  sum  of  the  input  signals  surpasses  a  certain  threshold,  the  neuron 
sends  an  action  potential  (AP)  at  the  axon  hillock  and  transmits  this  electrical  signal  along  the 
axon. 

The  fundamental  unit  of  the  biological  neural  network  is  called  a  neuron  or  nerve  cell.  Figure  12 
shows  a  schematic  of  the  structure  of  the  neuron.  The  main  body  of  cell  is  called  “soma”  where 
nucleus  is  located.  It  has  tree  like  fine  fibers  attached  called  dendrites.  These  dendrites  receive 
signals  from  other  neurons.  “Axon”  is  the  single  long  fiber  which  extends  from  the  soma,  which 
branches  into  strands  and  sub-strands  connecting  to  many  other  neurons  at  the  synaptic  junction. 
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Figure  11:  Biological  neurons. 


Axon  from 
another  neuron 


Each  neuron  accepts  stimuli  from  other  neighbouring  neurons  and  produces  an  output  as  soon  as 
the  overall  effect  of  the  input  stimulus  exceeds  the  threshold  limit  that  a  neuron  can  bear.  The 
connection  and  connection  strength  among  neurons  reflect  the  knowledge  pattern  of  human 
beings  (Barr  and  Kiernan,  1988).  A  neural  network  is  a  massively  parallel  distributed  processor 
that  has  a  natural  propensity  for  storing  experimental  knowledge  and  making  it  available  for  use. 

The  brain  is  a  collection  of  about  10  billion  interconnected  neurons  (Dowling,  2001).  Each 
neuron  is  a  cell  that  uses  biochemical  reactions  to  receive,  process  and  transmit  information.  A 
neuron’s  dendritic  tree  is  connected  to  a  thousand  neighbouring  neurons.  When  one  of  those 
neurons  fires,  a  positive  or  negative  charge  is  received  by  one  of  the  dendrites.  The  strengths  of 
all  the  received  charges  are  added  together  through  the  processes  of  spatial  and  temporal 
summation. 

An  Artificial  Neural  Network  (ANN),  as  described  in  Section  3.1.3,  is  an  information  processing 
paradigm  that  is  inspired  by  the  way  biological  nervous  systems  process  information.  Figure  13 
shows  an  artificial  neuron  structure.  In  this  diagram,  various  inputs  to  the  network  are  represented 
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by  the  mathematical  symbol,  Ij.  Each  of  these  inputs  is  multiplied  by  a  connection  weight 
represented  by  Wj.  In  the  simplest  case,  these  products  are  simply  summed,  fed  through  a  transfer 
function  to  generate  a  result,  and  then  produce  an  output.  This  process  lends  itself  to  physical 
implementation  on  a  large  scale  in  a  small  package.  This  electronic  implementation  is  still 
possible  with  other  network  structures  which  utilize  different  summing  functions  as  well  as 
different  transfer  functions. 

h 


Figure  13:  Neuron  in  artificial  neural  networks. 

These  networks  are  also  similar  to  the  biological  neural  networks  in  the  sense  that  functions  are 
performed  collectively  and  in  parallel  by  the  units,  rather  than  there  being  a  clear  delineation  of 
subtasks  to  which  various  units  are  assigned. 

4.2.2  Human  memory  models 

In  psychology,  memory  is  an  organism's  ability  to  store,  retain,  and  recall  information  (Neath  & 
Surprenant,  2002).  Traditional  studies  of  memory  began  in  the  fields  of  philosophy,  including  the 
development  of  techniques  aimed  at  artificially  enhancing  memory  functions.  The  late  nineteenth 
and  early  twentieth  century  placed  memory  within  the  paradigms  of  cognitive  psychology.  In 
recent  decades,  it  has  become  one  of  the  principal  pillars  of  a  branch  of  science  called  cognitive 
neuroscience,  an  interdisciplinary  link  between  cognitive  psychology  and  neuroscience. 

From  an  information  processing  perspective  there  are  three  main  stages  in  the  formation  and 
retrieval  of  memory: 

•  Encoding  or  registration  (receiving,  processing  and  combining  of  received  information); 

•  Storage  (creation  of  a  permanent  record  of  the  encoded  information);  and 

•  Retrieval,  recall  or  recollection  (calling  back  the  stored  information  in  response  to  some 
cue  for  use  in  a  process  or  activity). 

There  are  many  memory  models  proposed  including  the  Matrix  model  (Humphries  et  al.,  1989; 
Pike,  1984),  Search  of  Associative  Memory  (SAM),  Theory  of  Distributed  Associative  Memory 
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(TOD AM),  MINERVA  2,  Scale  Invariant  Memory  and  Perceptual  Learning  (SIMPLE)  (Brown  et 
al.,  2007),  etc.  This  section  reviews  three  typical  models:  SAM,  TODAM  and  MINERVA  2. 

4.2. 2.1  Search  of  Associative  Memory  (SAM) 

Search  of  Associative  Memory  (SAM)  model  (Raaijmakers  &  Schiffrin,  1981)  is  a  standard 
model  of  memory  that  employs  association.  Though  SAM  was  originally  designed  to  model 
episodic  memory,  its  mechanisms  are  sufficient  to  support  some  semantic  memory 
representations  as  well  (Kimball  et  al.,  2007).  In  SAM,  when  any  two  items  simultaneously 
occupy  a  working  memory  buffer,  the  strength  of  their  association  is  incremented.  Thus,  items 
that  co-occur  more  often  are  more  strongly  associated.  Items  in  SAM  are  also  associated  with  a 
specific  context,  where  the  strength  of  that  association  is  determined  by  how  long  each  item  is 
present  in  a  given  context.  In  SAM  memories  consist  of  a  set  of  associations  between  items  in 
memory  and  between  items  and  contexts.  The  presence  of  a  set  of  items  and/or  a  context  is  more 
likely  to  evoke  some  subset  of  the  items  contained  within  memory.  The  degree  to  which  items 
evoke  one  another — either  by  virtue  of  their  shared  context  or  their  co-occurrence — is  an 
indication  of  the  items’  semantic  relatedness.  The  Retrieving  Effectively  from  Memory  (REM) 
(Shiffrin  and  Steyvers,  1997)  is  another  model  in  the  SAM  family. 

4.2. 2. 2  Theory  of  Distributed  Associative  Memory  (TODAM) 

Theory  of  Distributed  Associative  Memory  (TODAM)  is  described  in  a  series  of  articles  by 
Murdock  and  his  colleagues  (Murdock  1982,  1983,  1997).  TODAM  is  of  interest  for  two 
reasons.  First,  it  stores  all  information  in  one  main  memory  vector  (the  length  of  which  is  the 
parameter  N).  As  such,  TODAM  uses  a  distributed  form  of  representation:  all  elements  are 
involved  in  representing  all  items.  Information  is  stored  using  convolution  and  retrieved  using 
correlation.  TODAM  thus  serves  as  an  existence  proof  that  serial  memory  phenomena  can  be 
modeled  successfully  using  only  one  memory  system  which  is  viewed  as  a  single  vector. 

Second,  TODAM  provides  an  elegant  solution  to  the  chaining  problem.  If  an  item  is  not  recalled, 
the  link  in  the  chain  is  missing;  in  the  typical  chain  model  of  memory,  recall  must  necessarily 
stop  at  this  point.  This  problem  has  limited  development  of  models  of  serial  order  for  years.  One 
of  the  contributions  of  TODAM  is  a  way  around  this  problem.  The  result  of  correlation  is  a  blurry 
but  potentially  interpretable  vector.  The  way  the  model  interprets  this  vector,  f ),  is  to  compute  the 
dot  product  between  the  recovered  vector  and  the  possible  candidates  and  if  this  value  is  within 
acceptable  limits,  then  the  item  is  interpreted  and  produced.  One  of  TODAM's  attractive 
properties  is  what  happens  if  the  blurry  vector  cannot  be  interpreted;  even  if  this  is  the  case,  f )  can 
be  used  as  the  cue  for  the  next  item  and  will  often  lead  to  successful  retrieval  of  the  next  item. 

4.2. 2. 3  MINERVA  2 

MINERVA  2  (Hintzman.  1984.  1986,  1988)  is  another  recognized  memory  model.  One  of  the 
main  goals  was  to  explain  memory  for  individual  experiences  (episodic  memory)  and  memory  for 
abstract  concepts  (generic  or  semantic  memory)  within  a  single  system.  This  version  implements 
some  of  the  simulations  reported  by  Hintzman. 
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This  version  of  MINERVA  2  concentrates  on  schema  abstraction,  recognition,  and  frequency 
judgments  and  is  best  thought  of  as  an  existence  proof:  the  program  proves  that  it  is  possible  to 
account  for  many  aspects  of  memory  for  individual  experiences  (i.e.,  episodic  memory)  and 
memory  for  abstract  concepts  (i.e.,  generic  or  semantic  memory)  within  a  single  system.  This 
does  not  prove  that  there  is  only  a  single  system;  rather,  it  proves  it  can  be  done. 

4.2.3  Analogy 

Analogy  has  long  been  recognized  an  a  important  component  of  intelligence  (Halford  et  al.,  2006, 
Binet  &  Simon,  1905/1980;  Piaget,  1950),  playing  a  significant  role  in  mathematics  (English  & 
Halford,  1995;  Polya,  1954),  science  (Tweney,  1998;  Holyoak  &  Thagard,  1995;  Dunbar  & 
Klahr,  1989),  politics,  art,  religion,  pedagogy,  communication,  humour  ,  and  law  (Holyoak  & 
Thagard,  1995).  Analogy  is  now  being  recognized  as  a  fundamental  process  in  natural  reasoning 
(Halford,  1992;  Hofstadter,  2001).  Its  basic  role  in  reasoning  is  indicated  by  the  fact  that  mental 
models,  which  are  important  to  some  theories  of  human  reasoning  (Johnson-Laird,  1983; 
Johnson-Laird  &  Byrne,  1991;  Polk  &  Newell,  1995)  and  cognitive  development  (Halford,  1993) 
are  essentially  based  on  processes  of  analogy.  Analogy  is  used  to  affect  transfer  between 
isomoiphic  tasks  (Halford,  Bain,  Maybery,  &  Andrews,  1998;  Novick,  1988;  Reed,  Ackinclose  & 
Voss,  1990;  Reed,  1987)  and  is  particularly  important  to  transfer  between  domains  (Gentner  & 
Gentner,  1983;  Gick  &  Holyoak,  1983).  The  literature  on  analogy  has  blossomed  since  1983  and 
there  are  a  number  of  volumes  devoted  to  it  (Holyoak  &  Barnden,  1994;  Gentner,  Holyoak  & 
Kokinov,  2001).  There  is  also  a  review  by  Holyoak  worth  noting  (2005). 

A  number  of  computational  models  of  analogy  have  been  developed.  These  include  Analogical 
Constraint  Mapping  Engine  (ACME)  (Holyoak  &  Thagard,  1989),  Structure-Mapping  Engine 
(SME)  (Falkenhainer,  Forbus,  &  Gentner,  1989);  Copycat  (Mitchell  &  Hofstadter,  1990), 
Incremental  Analogy  Machine  (IAM)  (Keane  &  Brayshaw,  1988),  Structured  Tensor  Analogical 
Reasoning  (STAR)  (Halford  et  al.,  1994;  Wilson  et  al.,  2001),  Learning  and  Inference  with 
Schemas  and  Analogies  (LISA)  (Hummel  &  Holyoak,  1997,  2003,  2005).  SME,  ACME  and 
LISA  are  summarized  by  Holyoak  (Holyoak,  2005)  and  an  overview  of  LISA  is  given  by 
Hummel  and  Holyoak  (Hummel  &  Holyoak,  2005).  Some  of  these  models,  ACME,  Copycat, 
STAR  and  LISA  are  neural  network  models,  and  can  also  been  seen  as  contributions  to  the  area 
of  symbolic  connectionism. 

Human  mental  models  for  deduction  can  be  based  on  analogy.  A  mental  model  is  iconic,  meaning 
that  there  is  some  structural  correspondence  between  the  model  and  the  content  of  the  problem. 
As  structural  correspondence  is  essentially  the  defining  property  of  analogies,  it  is  selected  or 
retrieved  from  memory.  As  an  example  of  the  way  analogy  could  be  used  to  generate  a  mental 
model,  consider  the  task  of  assessing  the  validity  of  the  inference,  p  — »  q,  ~p,  therefore  ~q.  An 
everyday  analog  could  be  used  such  as:  rain  implies  clouds,  no  rain,  therefore  no  clouds.  This 
makes  it  obvious  that  the  inference  is  not  valid,  and  is  a  natural  way  to  simplify  the  problem. 

Some  examples  of  applications  of  analogical  reasoning  consist  of  situation  assessment  Yang  et 
al.,  2006;  Louvieris  et  al.,  2006),  threat  assessment  /  Security:  foreign  policies  (Hehir,  2006; 
Markman  et  al.,  2003;  Knopf,  2002),  decision  support  (Louvieris  et  al.,  2006),  and  planning 
(Forbus  et  al.,  2004;  Cox  &  Veloso,  1997;  Haign,  et  al.,  1997). 
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4.2.4  Dual-process  thinking 


There  is  evidence  that  anatomically  distinct  regions  of  the  human  brain  are  involved  in  two 
processes:  a  low  process  and  a  high  process.  These  processes  directly  affect  human  reasoning  and 
thinking.  In  a  situation  where  there  was  a  conflict  between  valid  unbelievable  (high  process)  and 
invalid-believable  inferences  (low  process),  a  functional  Magnetic  Resonance  Imaging  (fMRI) 
study  showed  resolution  in  favour  of  valid  inferences  involving  the  right  inferior  prefrontal 
cortex,  but  resolution  in  favour  of  invalid  inferences  involving  the  ventral  medial  prefrontal 
cortex  (Goel  et  al.,  2000;  Goel  &  Dolan,  2004).  Based  on  the  results,  two  levels  of  reasoning  and 
thinking  have  been  identified  in  many  contexts  and  models  should  recognize  that  more  than  one 
type  of  reasoning  process  occurs  (Halford  et  al.,  2006).  Models  that  integrate  these  levels  offer 
the  best  potential  for  capturing  the  flexibility  and  robustness  of  thinking.  Table  4  summarizes 
distinctions  between  levels  of  cognitive  processes. 

Table  5:  Distinctions  between  levels  of  cognitive  processes  (Halford  et  al.,  2006). 


Low  Process 

High  Process 

References 

System  1 

System  2 

Evans  (2003) 

Implicit 

Explicit 

Reber,  1967 

Clark  &  Karmiloff-Smith,  1993 
Karmiloff-Smith,  1992 

Automatic 

Effortful/controllable 

Hasher  &  Zacks,  1979 

Low  processing  demands 

High  processing  demands 

Logan,  1979 

Schneider  &  Shiffrin,  1977 

No  volitional  or  adaptable 

Volitional 

Evans,  2003 

Without  training  involving  external 
input 

Responsive  to  verbal  instructions 

Clark  &  Karmiloff-Smith,  1993 

Associative 

Rule-based 

Slornan,  2002 

Driven  by  learning  or  innate  modules 

Based  on  thinking 

Evans,  2003 

Similarity  based 
Pragmatic/contextualized 

Rule  Logic/abstract 

Pothos,  2005 

Unconscious  except  for  final  product 

Conscious 

Norman,  1986 

Reber,  1992 

Evans,  2003 

Rapid 

Slow 

Norman,  1986 

Parallel 

Serial 

Evans,  2003 

Associative 

Relational 

Phillips,  Halford  &  Wilson,  1995 

Sub-symbolic 

Symbolic 

Smolensky,  1988 

Independent  of  language 

Related  to  language 

Accepted  by  many  authors 
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Low  Process 

High  Process 

References 

Independent  of  general  intelligence 

Related  to  general  intelligence 

Stanovich  &  West ,  2000;  Evans, 

2003 

Evolved  early 

Evolved  late 

Reber  &  Allen,  2000 

Shared  with  other  animals 

Uniquely  human  (or  uniquely 
primate) 

Evans  2003 

Dual  processes  in  reasoning  have  a  number  benefits: 

•  Two  systems  can  be  mutually  supportive,  so  System  2  can  facilitate  recognition  of 
similarity  by  System  1,  which  can  pre-process  information  for  use  by  System  2. 

•  Because  System  2  is  capacity  limited,  System  1  is  often  useful  in  conditions  of  high 
processing  load  in  providing  heuristic  solution  that  will  be  correct  most  of  the  time. 

Obviously,  in  the  modelling  and  simulation  of  human  thinking  and  reasoning,  the  dual-process 
approach  should  also  be  taken  into  account. 

4.2.5  Human  behaviour  moderators 

A  behavioural  moderator  is  a  condition  that  causes  a  change  to  an  individual  physiological  state 
or  environment  that  alters  the  behaviour  of  an  individual  in  any  way  (Pew  &  Mavor,  1998).  A 
number  of  performance  moderators  relevant  to  military-operator  simulation  have  been  identified 
in  literatures  (Pew  and  Mavor,  1998;  Ritter  and  Avraamides,  2000).  Pew  and  Mavor  classified 
behavioural  moderators  as  external  and  internal  moderators.  External  moderators  include 
physiological  stressors  (environment  (heat,  toxins,  noise,  vibration)  or  physical  workload  and 
fatigue)  and  cognitive  workload  stressors.  Internal  moderators  consist  of  intelligence,  expertise 
level  and  type,  cognitive  abilities,  personality,  and  emotions,  attitudes  and  cultural  values.  They 
believe  that  the  variables  they  identified  have  the  potential  to  impact  the  performance  of  soldiers 
and  commanders  in  the  field  and  therefore  should  be  considered  for  further  development. 

The  following  list  contains  some  example  models  /  references  of  behaviour  moderators: 

•  Emotion:  (Costa  &  McCrae,  1992;  Ortony  et  al,  1988;  Picard,  1997;  Henninger  et  al., 
2001,  2002,  2003;  Gratch  and  Marsella,  2001,  2004;  McRorie,  et  al.,  2009) 

•  Personality  and  Individual  difference  (Zachary  et  al.,  2005;  Eysenck,  1990;  Hudlicka  and 
Pfautz,  2002;  Silverman  and  Bharathy,  2005;  Read  and  Miller,  2002;  McKenzie  et  al., 
2001,  2003;  McRorie  et  al.,  2009;  Guo  &  Cain,  2009) 

•  Workload  and  fatigue  (Pfeiffer  et  al.,  1979;  Hendy  et  al.,  2000;  Stokes  &  Kite,  2000; 
Weaver  et  al.,  2000;  Ma  et  al.,  2009) 

•  Culture,  (Hofstede,  1980;  Klein  &  Klein,  2000;  Klein  2004). 


44 


DRDC  CORA  CR  2010-269 


4.3  Analysis  of  problems  in  current  wargame  replications 

LFORT,  with  many  years  of  experience  of  wargame  replications,  summarizes  the  features  in  both 
the  current  wargame  replicator  and  a  prospective  future  Al-driven  replicator,  as  shown  in  Table  6. 

Based  on  the  comparison  of  features  between  the  current  wargame  replicator  and  Al-driven 
replicator,  the  problems  in  current  wargame  replications  may  be  categorized  into  situation 
recognition  &  assessment,  planning  and  decision  making. 

The  following  list  includes  some  examples  of  human  intentions  related  to  situation/pattern 
recognition  and  assessment: 

•  Who  is  the  aggressor?  Who  is  on  the  defensive?; 

•  What  are  the  high  value  targets?; 

•  What  is  the  pattern  of  force  formation  (offence  +  defense)?; 

•  Is  one  side  retreating?; 

•  Is  on  force  trying  to  draw  the  other  into  a  particular  (vulnerable)  position,  or  force  them 
into  a  particular  position?  If,  so,  why?; 

•  Are  there  diversionary  tactics  at  play  (e.g.  enemy  suppression  to  ease  mobility)?; 

•  Are  some  units  dug  in  -  i.e.  prefer  to  fight  from  fixed  positions?; 

•  Are  they  defending  something  (i.e.,  convoy,  building)?;  and 

•  What  advantage  is  one  side  trying  to  gain  over  the  other  (e.g.,  high  ground,  covered 
positions,  buildings,  etc.)? 

Some  instances  associated  with  planning  and  decision  making  may  be 

•  Quality  or  value  of  courses  of  action; 

•  Assess  courses  of  action;  and 

•  Select  courses  of  action. 

Obviously,  the  Al-driven  replicator  should  provide  approaches  and  means  to  support  various 
human  intentions  in  wargame  situation  assessment,  knowledge  /  pattern  discovery,  planning  and 
decision  making. 
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Table  6:  Features  of  Current  Replicator  vs.  AI-Driven  Replicator 


Feature 

Current  Replicator 

Al-driven  Replicator 

Input 

CAEn  game  files:  routes,  aim  zones,  activities, 
orders,  arcs,  speed,  rules  of  engagement,  damage 
taken,  entity  state 

CAEn  probability  tables  (see  next  page) 

Aramis  game  files  (common  format  for  supported  wargame  systems):  routes,  speed,  shots 
fired,  damage  taken,  entity  state. 

Common  probability  tables  (see  below) 

Time-segmented  interactor  description  of  goals,  tactics,  movement  patterns,  rules  of 
engagement,  etc.. 

Output 

CAEn  game  files:  routes,  aim  zones,  activities, 
arcs,  speed,  damage  taken,  entity  state 

Aramis  game  files 

Additional  Information:  aim  zones,  activities,  arcs,  record  of  goals,  decisions  and  reasons 
behind  decisions  (entity  state). 

Entity  Movement 

Constrained  to  follow  exact  path  defined  in  game 

Based  on  entity  decisions  derived  from  interactor  description  of  movement  patterns 

Entity  Activities 

Activity  sequence  recorded  from  game.  Replicator 
reuses  exact  activity  sequence  of  game 

Goal-oriented  and  situation  dependent  activity  sequence  provided  by  interactors 

Entity  Interactions 

As  recorded  from  game  with  certain  limitations 

Depend  on  circumstances  as  defined  by  the  interactors. 

Probability  Tables  (eg..  Uses  CAEn  tables  Use  external  tables  common  to  all  wargames.  Must  decide  on  a  baseline  (e.g.,  JCATS  tables), 

hit,  kill,  detect) 


Terrain 


Terrain  not  taken  into  account 


GIS-based  terrain  features  taken  into  account,  based  on  interactors  description  of  how  terrain 
enters  into  decision  making. 


Exploration  of  the  Space  Limited.  Replications  strongly  polarized  to  the 
of  Possibilities  seed  game.  No  fitness  criteria  applied. 


Robust.  Wider  possibilities  explored  by  virtue  of  added  decision  points.  Fitness  criteria  applied 
(e.g.,  culling). 


Number  of  Interactive 
Games  Needed  to  Support 
Analysis 


At  least  four  (4)  depending  on  the  scenario,  but  Less  games  should  be  able  to  explore  more  possibilities.  Therefore,  expect  adequacy  with 
more  are  preferred.  Typically  20  replications  are  fewer  games  and  more  replications, 
produced  per  game. 


Length/Complexity 

Games 


of  Short/Simple.  Otherwise,  replication  results  can  Long/Complex.  Entities  consistently  make  similar  decisions  compared  to  ‘realistic’  interactors, 
become  nonsensical. 


4.4  An  Al-driven  wargame  replication  system 


This  section  describes  a  mid/long-term  solution  to  solve  all  kinds  of  problems  in  current  wargame 
replications,  which  uses  a  unified  memory  representation  structure,  integrates  results  from  human 
science  and  provides  a  mechanism  for  future  adaptability  of  problem  domains. 

4.4.1  Objectives 

The  objectives  of  the  proposed  solution  are  the  development  of  a  system  that: 

•  Absorbs  results  from  human  science:  memory,  ANN,  analogy,  and  dual-processes 
thinking; 

•  Defines  a  unified  memory  representation  structure  for  re-use  of  knowledge; 

•  Integrates  existing  recognized  approaches  and  tools; 

•  Supports  multiple  approaches  for  various  problem  areas; 

•  Provides  means  for  modelling  human  behaviour  moderators  ; 

•  Characterizes  various  cognitive  functions:  e.g.  situation  assessment,  planning,  decision 
making  and  learning;  and 

•  Recommends  adaptable  approaches  for  various  problems  in  wargame  replications. 


4.4.2  High  level  solution 

This  proposed  architecture  is  shown  in  Figure  14,  which  consists  of  an  AI  Replication  Engine,  a 
Post-Game  Analysis  Toolkit  (PGAT)  and  a  PGAT  Interface. 


The  AI  Replication  Engine  provides  various  AI  approaches  for  problem  solving  and  reasoning  in 
wargame  replications.  It  comprises  a  Working  Memory,  Regular  Thinking  Layer,  Meta  Layer  and 
Motor  In  terface. 


The  Working  Memory  component  is  a  sensing  interface  to  acquire  external  data,  and  the  Motor- 
Interface  transmits  the  result  of  problem  solving  or  reasoning  to  external  PGAT  modules  for 
further  use. 
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PGAT  Interface 


AI  Replication  Engine 


AI  Replicated 


Working 

Memory 


Meta  Laver: 

Regular  Thinking  Layer: 


Figure  14:  Al-Driven  Wargame  Replication  System. 

The  Regular  Thinking  Layer  is  the  kernel  in  the  structure  with  three  sub-components:  Long-Term 
memory,  Dual  Process  Approaches  and  Approach  Adaptor.  The  Long-Term  Memory  is  a  unified 
knowledge  representation  structure  defined  by  combining  the  features  from  human  science 
including  memory  models,  analogical  models,  dual  process  model  and  artificial  neural  networks. 
The  Dual  Process  Approach  Base  integrates  multiple  methods  related  to  both  abstract  and 
associative  levels.  Typical  examples  at  the  abstract  level  may  be  the  rule-based  reasoning  and 
fuzzy  computing,  and  some  representative  associative  methods  include  human  memory  models, 
Bayesian  networks  and  neural  networks.  The  Approach  Adaptor  is  used  to  recommend  better 
methods  for  a  given  problem  category.  For  example,  if  an  interactor  wants  to  assess  the  value  of  a 
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fighter,  the  system  checks  all  methods  in  the  system  for  evaluating  the  values  of  objects,  and  finds 
that  the  Utility  method  is  the  best  one  among  three  existing  approaches  in  the  system:  Bayesian 
networks,  Utility  and  Fuzzy  Logic,  and  then  the  system  selects  the  Utility  method  to  evaluate  the 
value  of  a  fighter  and  back  the  results  to  the  user. 

The  Meta  Layer  in  the  architecture  is  important  to  support  human  behaviour  moderators.  The 
basic  assumption  here  is  that  human  behaviour  moderators,  e.g.  personality,  emotion  and 
workload,  affect  the  process  and  results  of  human  thinking  and  reasoning.  The  Meta  Layer  and 
Regular  Thinking  Layer  work  together  to  enhance  the  variability  and  human-like  behaviour  in  the 
proposed  solution.  The  PGAT  Interface  in  the  structure  is  responsible  for  data  transmission  and 
format  conversion  between  PGAT  and  the  AI  replication  engine.  PGAT,  as  explained  in  Section 
2,  collects  data  from  wargames  in  Synthetic  Environments,  e.g.  CAEn,  stores  the  results  of 
simulation  to  the  central  database,  analyzes  data,  displays  data,  provides  queries  and  interacts 
with  the  AI  Replication  Engine  for  intelligent  behaviour.  By  combining  the  results  from  the  AI 
Replication  Engine,  a  ‘template’  Initial  Wargame  (or  wargames)  plus  supplementary  data  (all 
stored  in  the  VDB),  a  validated  set  of  Replicated  Wargames  are  generated  and  stored  in  the  VDB 
for  later  analysis  and  visualization  via  PGAT3. 

4.4.3  Features  of  the  proposed  architecture 

Compared  to  other  systems,  the  significant  features  of  the  proposed  solution  include 

•  Human  science  oriented  representation  to  enhance  reliability  and  human-like  behaviour; 

•  Re-use  of  knowledge  to  increase  system  efficiency; 

•  Multiple  approaches  to  widen  the  system  flexibility; 

•  Human  behaviour  moderators  to  characterize  variability  and  individual  differences;  and 

•  Learning  ability  to  recommend  suitable  approaches  and  enhance  adaptability. 


4.4.4  Potential  application  areas 

Some  potential  application  areas  of  the  proposed  solution  consist  of 

•  Al-driven  wargame  replications; 

•  Virtual  interactors  or  operators  in  wargame  replications  and  various  CGFs; 

•  Simulated  adversary; 

•  Virtual  training  partners:  pilots,  co-pilots,  landing  security  officers,  etc.; 

•  Cognitive  functions  for  generic  problem  solving  in  wargames  and  CGFs: 


3  Ideally,  it  would  be  advantageous  to  have  the  AI  Replication  engine  interact  directly  with  the  synthetic 
environment  as  interactors  would  to  produce  new  ‘games’.  However,  that  is  not  possible  with  the  current 
systems  employed.  Therefore,  the  AI  engine  must  draw  from  a  generic  representation  of  the  synthetic 
environment  based  on  data  in  the  VDB  (entity  capabilities,  entity  intent,  events,  probability  tables,  GIS 
terrain  features,  etc.).  Note  that  it  may  be  possible  to  specify  an  existing  ‘standard  replication  environment’ 
(such  as  JCATS)  where  artificial  interactors  generated  by  the  AI  engine  could  play  out  the  replications  and 
generate  the  required  data. 
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o  Situation  awareness; 
o  Decision  making  ; 
o  Planning;  and 

o  Knowledge  /  pattern  discovery,  or  learning  . 

•  Command  and  control,  search  and  rescue,  training,  M&S,  air  traffic  control,  etc. 
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5  System  scoping 


This  section  provides  an  estimate  pertaining  to  the  overall  level  of  effort  and  scope  related  to 
implementing  the  proposed  AI  driven  wargame  replication  system.  A  work  plan  is  proposed  with 
an  initial  phase  where  a  detailed  conceptual  model  of  the  AI  driven  wargame  replication  system  is 
developed.  Following  this  definition  phase  an  incremental  development  effort  is  proposed  that 
builds  from  an  initial  capability  to  the  full  objective  capability.  Significant  investigation  is  still 
required  to  fully  define  the  requirements  for  the  wargame  replication  system.  For  this  reason,  the 
estimates  contained  in  this  section  should  be  considered  preliminary  in  nature. 

5.1  Work  tasks  and  efforts 

Based  on  the  domain  cognitive  analysis  of  wargame  replications,  four  (4)  main  categories  of  tasks 
in  the  mid/long-term  solution  described  in  Section  4  are  identified.  The  development  effort 
proposed  is  an  iterative,  incremental  process  involving  categories  of  work  indentified  as  follows: 

•  Model  development:  9  person-months 

•  System  implementation:  19-46  person-months  depending  on  selected  tools  and  desired 
capability 

•  Application  development:  7  person-months 

•  Verification  &  validation:  6  person-months 

5.1.1  Model  development  (9  person-months) 

More  effort  is  required  to  define  the  requirements  for  the  wargame  replicator  in  more  detail. 
Based  on  these  system  requirements,  a  more  detailed  conceptual  model  of  the  system  must  be 
developed.  Conceptual  model  development  is  the  first  task  in  the  proposed  development  effort. 
The  following  Table  7  describes  the  main  sub-tasks  and  efforts  in  model  development: 

Table  7:Sub-tasks  in  model  development. 


Sub- Tasks  in  Model 
Development 

Efforts 

(person- 

month) 

Details  in  Wargame  Replicator 

Objectives  and  scope 
identification 

0.5 

Based  on  analysis  of  human  interactor  intent,  define  goals 
and  objectives; 

Define  the  boundaries  of  the  project. 

Typical  scenario 
development 

0.5 

Develop  typical  scenarios  that  span  the  experimentation 
space  that  the  replicator  will  need  to  address,  e.g. 
consisting  of  both  BLUE  and  RED  sides  involved  in  an 
urban  battle. 
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Identification  of 
requirements 

1.0 

Based  on  current  and  anticipated  wargame  needs,  define 
detailed  requirements  of  Al-driven  replicator. 

Cognitive  task  analysis 

0.5 

Analyze  a  collection  of  previous  or  existing  wargame 
scenarios; 

Generalize  wargame  goals,  tasks  and  activities. 

Identification  and 
categorization  of  user 
intentions 

0.5 

Gather,  analyze  and  categorize  intentions  of  human 
inter  actors  /  commanders  of  war  games. 

Component  and 
relationship  analysis  of 
interactor  intentions 

0.5 

For  each  category  of  problems,  identify  main  components 
and  relationships  between  them,  for  example,  situation 
recognition:  various  components  and  relationships  for 
recognizing  “aggressor”  and  “defender”,  values  of 
targets,  pattern  of  force  formation  (offence  +  defense), 
entity  intentions  (retreating?,  forcing  an  entity  to  do 
something  (e.g.  to  a  particular  position),  tactics 
recognition  (diversionary  tactics:  e.g.  Suppression  to  ease 
mobility),  unit  preferences  (prefer  to  fight  from  fixed 
positions),  attacking/defending  recognition,  units’ 
advantages/disadvantages  recognition; 

For  instance,  planning  &  decision  making:  quality  of 
courses  of  action,  selection  course  of  action;  decision 
evaluation,  &  decision  selection. 

Memory  structure 
modeling 

Declarative  memory 
modeling 

1.5 

Entities  and  facts  representation  (0.5  months): 
Representation  of  aim  zone,  arcs,  damage  taken,  entity 
state,  routes,  speed,  shots  fired,  etc.; 

Goals,  decisions,  reasons  behind  decisions; 

Common  Probability  Tables:  hit,  kill,  detect,  etc.; 

Terrain  DB:  entities  &  features,  position  relationships, 
values  of  entity  features  to  tactics:  defending,  offending; 

Procedural  memory 
modeling  (tasks  and 
activities) 

Procedures  /  tasks  representation  (0.75  months): 

Movement  patterns,  rules  of  engagements,  production 
rules,  algorithms; 

Courses  of  action,  activity  sequences  and  tasks. 

Dual-process  approach 
identification  and 
evaluation: 

Abstract 

Associative 

1.0 

Identify  abstract  approaches  for  rules  or  patterns  in 
wargame  decisions,  planning  or  situation  assessments,  & 
unit  movements;  Identify  methods  for  wargame  goal 
networks,  task  networks,  courses  of  action,  activities, 
situation  elements  and  relationships  networks;  associative 
networks  for  wargame  pattern  recognition,  knowledge 
discovery  &  other  cognitive  functions. 
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Modeling  of  system 
adaptability 

0.5 

Application/scenario-type-oriented  approach  evaluation; 
e.g.  method  measurement  and  comparison  for  the 
evaluation  of  wargame  targets. 

Protocol  development  for 
component  interactions 

0.5 

Define  and  develop  interaction  requirements  between 
various  AI  &  wargame  components. 

Behaviour  moderator 
model  development: 

Factor  identification 
Models  categorization 
Effect  exploration 

Model  integration 

1.0 

Identify  various  moderator  factors  of  human  interactors 
and  virtual  operators  in  wargames,  e.g.  emotion  and 
personality; 

Categorize  them  based  on  common  features  and 
associations; 

Explore  effects  of  behaviour  moderators  to  wargame 
interactors  performance; 

Integrate  important  models. 

Options  Analysis 

1.0 

Cross  reference  the  system  requirements  with  the 
capabilities  of  existing  cognitive  modelling  and  artificial 
intelligence  application  and  existing  ontologies  and  other 
knowledge  bases  to  inform  which  aspects  of  the  system 
will  be  formed  by  the  integration  of  existing  applications, 
and  which  aspects  of  the  system  will  have  to  be 
developed. 

Develop  Implementation 
plan 

1.0 

Build  on  the  documentation  of  the  system  requirements 
and  the  conceptual  model  and  develop  the 
implementation  plan  that  provides  the  detailed  work  plan 
for  the  design,  development  testing  and  deployment  of 
the  wargame  replication  system 

5.1.2  System  implementation  (19  -  46  person-months) 

The  time  range  in  the  system  implementation  may  range  from  19-46  person-months,  depending 
on  the  selected  tools  to  be  integrated  into  the  proposed  solution  and  the  final  scope  of  the  system 
requirements.  The  following  Table  8  shows  the  sub-tasks,  efforts  and  details  in  wargame 
replicator  in  system  implementation.  Please  note  that  while  the  work  tasks  are  laid  out  in  a 
waterfall  fashion  in  table  8,  an  incremental  spiral  development  approach  is  recommended  to 
provide  capability  in  an  incremental  fashion  and  for  risk  reduction. 

Table  8:  Sub-tasks  in  system  implementation. 


Sub- Tasks  of  System 
Implementation 

Efforts 

(person- 

month) 

Details  in  Wargame  Replicator 

Requirement  specification 

1.5  -3.0 

From  software  point  of  view,  develop  detailed 
requirements  of  Al-driven  replicator  from  the  system 
requirements  and  conceptual  model. 

Scope,  boundary  and  risk 
identification 

0.5  -2.0 

Analyze  and  identify  software’s  scope,  boundary  and 
risks. 
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Decomposition  of  detailed 
functionality 

2.0 -6.0 

Itemize  functionality  of  software  modules. 

Architecture  &  system 
design 

actor  identification 

o 

\D 

1 

q 

rd 

Identify  human  interactors  at  various  wargame  unit 
levels,  interactions,  external  hardware  and  software, 
network  components,  protocols,  &connections; 

Use  case  development 
Interaction  development 

Categorize  detailed  functionality  &  interactions  of 
system  components; 

Class  identification  and 
design 

Define  logic  structures  of  software; 

Significant  algorithms  and 
activities 

Main  algorithm  developments. 

Implementation: 

Basic  memory  knowledge 
base  development: 
declarative  &  procedural  (3 
-  4  months) 

12.0- 

28.0 

Implement  various  representations  of  entities 
relationships  and  associations:  e.g.  aim  zones,  arcs, 
damage  taken,  entity  state,  routes,  speed,  shots  fired, 
goals,  context,  decisions,  common  probability  tables, 
terrains. 

Realize  dynamic  procedures  and  tasks:  movement 
patterns,  rules  of  engagements,  production  rules, 
algorithms,  courses  of  action,  activity  sequences,  and 
other  relevant  tasks. 

Integrate  existing  tools  (4- 
14  months) 

Choose  existing  tools  to  integrate:  e.g.  Soar,  ACT-R, 
Bayesian,  ANN,  CBR,  LAMP  etc.; 

Integrate  the  selected  tools; 

Protocol  development  for 
interaction  between  tools  (1 
-  2  months) 

Analyze  wargame  tasks  and  assign  wargame 
intelligent  tasks  to  various  tools. 

Implement  interactions  between  tools; 

Working  memory  and  motor 
interface  (1-2  months) 

Analyze  data  and  constraints  in  interactions  between 

AI  replication  engine  and  PGAT ; 

Define  data  formats  and  data  exchanges  between  AI 
engine  and  PGAT ; 

Develop  data  structures  and  integrate  them  into  the 
system; 
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Interface  to  synthetic 

Possibly  define  and  realize  an  interface  to  a  specified 

environment  (1-2  months) 

standard  synthetic  environment  to  host  the  ‘Al- 
interactive’  games  (wargames  utilizing  artificial 
interactors),  e.g.  JCATS; 

Interface  to  PGAT  (1-2 

Implement  input/output  and  communication  interfaces 

months) 

to  PGAT  data  bases; 

Tests  (1-2  months) 

Verification  of  the  implemented  software. 

5.1.3  Application  integration  and  configuration  (7  person-months) 

As  increments  of  the  AI  driven  wargame  replication  system  are  developed,  they  will  be  integrated 
with  DRDC  CORA’s  PGAT  infrastructure,  and  configured  with  the  necessary  data  to  support  the 
required  replications.  The  sub-tasks,  efforts  and  details  in  application  integration  and 
configuration  are  shown  in  Table  9. 

Table  9:  Sub-tasks  in  application  development. 


Sub-Tasks  in  Application 
Development 

Efforts 

(person- 

month) 

Details  in  Wargame  Replicator 

Scenario  analysis  and 
definition 

1.0 

Define  and  develop  a  group  of  typical  wargame 
scenarios. 

I/O  data  identification 

0.5 

Identify  data  from  the  wargame  scenarios. 

Scenario  development  in 

SE 

2.0 

Establish  scenarios  in  synthetic  environment:  CAEn  / 
JCATS. 

Integrate  with  SE  &  PGAT 
for  applications 

3.5 

Complete  the  scenarios  implementation  in  Synthetic 
Environment,  PGAT  and  AI  replication  engine; 

Implement  interactions  between  PGAT,  CAEn/JCATS 
and  AI  replication  engine. 

5.1.4  Verification  and  validation  (6  person-months) 

As  an  ongoing  process  with  the  development  of  replicator  increments,  and  as  the  last  phase  of  the 
work  effort,  Verification  and  Validation  (V&V)  is  a  critical  component  of  the  proposed  work 
plan.  The  sub-tasks  of  the  V&V  effort  are  described  in  Table  10. 
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Table  10:  Sub-tasks  in  verification  &  validation. 


Sub- Tasks  in  Verification 
&  Validation 

Efforts 

(person- 

month) 

Details  in  Wargame  Replicator 

Verification  of  software 
system 

1.0 

Software  packages  and  system  testing. 

Identification  of  validation 

resources 

0.5 

Classify,  recognize  and  find  various  human  and  other 
resources  required  by  validation. 

Scenario  identification  & 
selection 

1.0 

Develop  wargame  scenarios  for  validation. 

Development  of  validation 
process  and  methods 

0.5 

Work  out  methodology  for  validation  of  Al-driven 
replicator. 

Validation  conduction 

2.0 

Validate  the  Al-driven  replicator. 

Result  handling  of 
validation 

1.0 

Analyze  and  summarize  validation  results. 
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6  Recommendations 


6.1  Short-term  experimental  solutions 

There  are  many  HBR  and  AI  tools  as  described  in  Section  3,  but  there  is  no  single  tool  /  approach 
that  has  the  capacity  to  span  the  problem  space  of  wargame  replications.  However,  each  tool  or 
approach  has  its  own  features  and  strengths  and  can  be  used  for  some  limited  range  of 
experiments  /  tests  in  wargame  replications.  Based  on  the  categories  of  the  problems  in  wargame 
replications,  the  following  Table  11  recommends  some  tools  or  approaches  to  solve  problems  in 
related  categories. 

Table  11:  Recommendation  for  short-term  solutions. 


Problem  Category 

Approaches  /  Tools  Recommended 

Efforts  for  Integration 
and  simple  example 
development 

(person-month) 

Situation/threat 

assessment. 

Pattern 

recognition/classification 

Bayesian  tools,  neural  networks 

3.0 -5.0 

Rule-based  tools:  ACT-R,  Soar,  Common 
Lisp,  &  Prolog 

3.0 -5.0 

Others:  HMM,  SVM,  fuzzy,  K-nearest 
neighbor,  etc 

3.0 -5.0 

Planning 

Rule-based:  Soar,  ACT-R 

3.0 -5.0 

Genetic  algorithms 

3.0 -5.0 

BD1 

3.0 -5.0 

Others:  Bayesian,  ANN,  CBR 

3.0 -5.0 

Decision  Making 

Decision  Trees,  Utility,  RPDM,  Bayesian 
networks 

3.0 -5.0 

Rule-based:  Soar,  ACT-R,  Lisp,  Prolog 

3.0 -5.0 

Others:  CBR,  fuzzy,  ANN,  etc. 

3.0 -5.0 

Generic  Cognitive 
functions 

ACT-R,  Soar,  LAMP 

3.0 -5.0 
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6.2  Mid/long-term  generic  solution 

The  mid/long-term  solution  described  in  Section  4  is  a  human-science-oriented,  knowledge- 
reusable,  integrated  and  adaptable  system.  The  main  purpose  of  the  proposal  is  to  meet  the  needs 
of  current  and  future  wargame  replications.  Combining  various  results  from  human  science  makes 
the  system  have  a  solid  cognitive  foundation.  A  unified  memory  structure  that  supports  multiple 
approaches  provides  the  capability  to  re-use  the  same  group  of  knowledge/information/data  for 
different  experiments  and  performance  evaluation.  Integrating  multiple  approaches  is  to  extend 
the  categories  of  problem  solving  for  future  wargame  replications,  e.g.  from  low-level  situation 
perception/recognition  to  high-level  decision  making  and  planning.  Furthermore,  modelling 
human  behaviour  moderators,  and  selecting/recommending  appropriate  approaches  are  able  to 
improve  human-like  variability  and  system  adaptability. 


The  main  tasks,  efforts  and  outputs  in  the  mid/long-term  solution  are  summarized  in  Table  12. 
Table  12:  Recommendation  for  mid/long-term  solutions. 


Task 

Sub-task 

Efforts 

(person- 

month) 

Outputs 

Unified  memory 
/KR  and 
Feasibility  Proof 

Feasibility  model 
development 

3.0 

Memory  representation  structure 
development:  A  re-usable  memory 
representation  structure:  aim  zones,  arcs, 
damage  taken,  entity  state,  routes,  speed,  shots 
fired,  goals,  decisions,  reasons  behind 
decisions,  common  probability  tables,  terrain 
DB,  dynamic  procedures:  movement  patterns, 
rules  of  engagements,  courses  of  action, 
activity  sequences,  rules,  tasks; 

Unified  memory 
development 

4.0 

Initial  integration 
for  abstract 
process  approach 

3.5 

Example 

development 

1.5 

Integration  of  memory  structure  &  abstract 
approach; 

Wargame  application  development: 
Demonstration  of  initial  scenario:  typical 
scenarios  plus  AI  functions  for  situation 
assessment  or  decision  support. 

Main 

Architecture 

2.0 

System  architecture  development:  software 

functionality 

design 

logic  components,  relationships  and 

development 

Abstract 

7.0- 

organization  for  various  wargame  entities, 

COA,  decisions,  terrains  and  Al-related 

approaches 

12.0 

components; 

Associative 

7.0- 

Typical  functions  developments:  Typical 

abstract  &  associative  functions  development: 
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approaches 

12.0 

for  pattern  recognition,  knowledge  discovery, 
situation  assessment,  planning  and  decision 
support:  recognizing  aggressor,  defender, 
target  values,  pattern  of  formation;  retreating 
recognition;  adversary  intentions  (one  forces 
another  to  do  something;  defending 
something),  tactics  recognition;  preferences, 
advantages  evaluation  (high  ground,  covered 
positions,  buildings,  etc.); 

Typical  wargame  applications  development 
and  demonstration. 

Extended 

applications 

4.0  -  6.0 

Advanced 

adaptable 

functions 

Meta  Layer 
model 

development 

O 
o 6 

p 

Adaptive  functions  development:  develop 
measurements  to  evaluate  various  approaches 
for  problem  categories  in  wargame 
replications; 

Adaptive  applications  implementation:  build 
scenarios  with  advanced  adaptability 
functions;  verify  &  validate  the  adaptive 
functions. 

Learning  and 
Adaptive 
functions 
implementation 

5.0- 

16.0 

Total 

41.0- 

68.0 

(See  Section  4  for  details). 
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AI 

Artificial  Intelligence 
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